Screen Capture in SIlverlight 4.0

Screen Capture in SIlverlight 4.0

Recently I have been doing a very sophisticated Silverlight project. In this project there was a need to capture the screen and to save the image in a data base. The known way to capture a screen is to use a WriteableBitmap class instance (See Jeff Prosise blog about this feature that was added in SL 3.0). To capture the screen we use the code from http://stackoverflow.com/questions/1139200/using-fjcore-to-encode-silverlight-writeablebitmap 

The following code is a service that captures the screen image:

 

    public classSnapshotService : ISnapshotService
    {
        public byte[] Capture()
        {
            var bitmap =  newWriteableBitmap(Application.Current.RootVisual, null);
           
            returnSaveToArray(bitmap);
        }

        private static byte[] SaveToArray(WriteableBitmap bitmap)
        {
            int width = bitmap.PixelWidth;
            int height = bitmap.PixelHeight;

             const int bands = 3;
            var raster = new byte[bands][,];

            //Code From http://stackoverflow.com/questions/1139200/using-fjcore-to-encode-silverlight-writeablebitmap
           for(int i = 0; i < bands; i++)
            {
                raster[i] = new byte[width, height];
            }


            for(int row = 0; row < height; row++)
            {
                for(int column = 0; column < width; column++)
                {
                    int idx = (width * row) + column;
                    if(idx > 0)
                    {
                        //NOTE: this might fail due to pixels which might be considered as 'not trusted' and
                       // therefore the access to these pixel will be denied
                        // "WriteableBitmap has protected content. Pixel access is not allowed."
                       int pixel = bitmap.Pixels[idx];

                        raster[0][column, row] = (byte)(pixel >> 16);
                        raster[1][column, row] = (byte)(pixel >> 8);
                        raster[2][column, row] = (byte)(pixel);
                    }
                }
            }
 
            var model = new ColorModel { colorspace = ColorSpace.RGB };
            var img = new FluxJpeg.Core.Image(model, raster);

            //Encode the Image as a JPEG
            var stream = new MemoryStream();
            var encoder = new JpegEncoder(img, 60, stream);
            encoder.Encode();

            //Back to the start
            stream.Seek(0, SeekOrigin.Begin);

            //Get teh Bytes and write them to the stream
            var binaryData = new Byte[stream.Length];
            stream.Read(binaryData, 0, (int)stream.Length);
            return binaryData;
        }


        public BitmapImage Decode(byte[] image)
        {
            if (image == null) return null;
            var b = new BitmapImage();

            using (var stream = new MemoryStream(image))
            {
                b.SetSource(stream);
                return b;
            }

        }
    }

The main problem with this method is that it is not always work. We use Bing map control and Media Elements in our Visual Tree, This causes the line   int pixel = bitmap.Pixels[idx]; to throw Security exception. The exception is there to protect against violating of Digital Rights (DRM) and it is by design. See this link to read more about the problem. First I thought that the problem comes from the fact that the GIS information and the video stream sources come from a site which is not the same as the Silverlight application site. a Cross-Domain policy file should solve this kind of problems. Cross Domain Policy file should come from the site that contains the elements that get rendered on the visual tree. In this case we cannot control the source site and further investigating proved that cross-domain file is useless for this problem. Since the project was a prototype and we ran out of time a radical solution emerged. I decided to use another known Silverlight application that will run on the client and will take a screen capture of the IE tab from the outside. To capture an IE tab, I had to find the IE tab Windows handle, I played with Spy++, understood the relationship between windows under IE, found out that the Window class of Silverlight is "MicrosoftSilverlight" and created the FindFrameWindow method. The rest is just a Win32 BitBlt and WinForm Bitmap/jpeg support:

using System; using System.Drawing; using System.Drawing.Imaging; using System.Runtime.InteropServices; using System.Text; using System.IO;  namespace G2ScreenCapturer {     class Gdi32     {         [DllImport("GDI32.dll")]         public static extern bool BitBlt(IntPtr hdcDest, int nXDest, int nYDest, int nWidth, int nHeight, IntPtr hdcSrc, int nXSrc, int nYSrc, int dwRop);         [DllImport("GDI32.dll")]         public static extern IntPtr CreateCompatibleBitmap(IntPtr hdc, int nWidth, int nHeight);         [DllImport("GDI32.dll")]         public static extern IntPtr CreateCompatibleDC(IntPtr hdc);         [DllImport("GDI32.dll")]         public static extern bool DeleteDC(IntPtr hdc);         [DllImport("GDI32.dll")]         public static extern bool DeleteObject(IntPtr hObject);         [DllImport("GDI32.dll")]         public static extern int GetDeviceCaps(IntPtr hdc, int nIndex);         [DllImport("GDI32.dll")]         public static extern IntPtr SelectObject(IntPtr hdc, IntPtr hgdiobj);     }      class User32     {         public delegate bool EnumFunc(IntPtr hWnd, uint lParam);          [DllImport("User32.dll")]         public static extern IntPtr GetWindowDC(IntPtr hWnd);          [DllImport("User32.dll")]         public static extern int ReleaseDC(IntPtr hWnd, IntPtr hDc);          [DllImport("User32.dll")]         public static extern bool EnumChildWindows(IntPtr hWndParent, EnumFunc ef, uint lParam);          [DllImport("User32.dll", CharSet = CharSet.Auto)]         public static extern int GetClassName(IntPtr hWnd, StringBuilder text, int nMaxCount);          [DllImport("User32.dll")]         public static extern IntPtr FindWindow(string lpClassName, string lpWindowName);      }      class ScreenCapturer     {         public static byte [] CaptureScreen()         {              var hWndSrc = FindFrameWindow();             var hdcSrc = User32.GetWindowDC(hWndSrc);             var hdcDest = Gdi32.CreateCompatibleDC(hdcSrc);             var hBitmap = Gdi32.CreateCompatibleBitmap(hdcSrc,                                                        Gdi32.GetDeviceCaps(hdcSrc, 8), Gdi32.GetDeviceCaps(hdcSrc, 10));             Gdi32.SelectObject(hdcDest, hBitmap);             Gdi32.BitBlt(hdcDest, 0, 0, Gdi32.GetDeviceCaps(hdcSrc, 8),                          Gdi32.GetDeviceCaps(hdcSrc, 10), hdcSrc, 0, 0, 0x00CC0020);             var result = GetImage(hBitmap);             Cleanup(hdcSrc, hBitmap, hdcSrc, hdcDest);             return result;         }          private static void Cleanup(IntPtr hWndSrc, IntPtr hBitmap, IntPtr hdcSrc, IntPtr hdcDest)         {             User32.ReleaseDC(hWndSrc, hdcSrc);             Gdi32.DeleteDC(hdcDest);             Gdi32.DeleteObject(hBitmap);         }          private static byte [] GetImage(IntPtr hBitmap)         {              var capture =                 new Bitmap(Image.FromHbitmap(hBitmap),                            Image.FromHbitmap(hBitmap).Width,                            Image.FromHbitmap(hBitmap).Height);              var temporaryimageFile = Path.GetTempFileName();             capture.Save(temporaryimageFile, ImageFormat.Jpeg);              byte[] result;              using (var file = File.OpenRead(temporaryimageFile))             {                 result = new byte[file.Length];                 file.Read(result, 0, (int)file.Length);             }             File.Delete(temporaryimageFile);             return result;         }          private static IntPtr FindFrameWindow()         {             IntPtr frameHWnd = IntPtr.Zero;              var ieWnd = User32.FindWindow("IEFrame", "Title - Windows Internet Explorer");              User32.EnumChildWindows(ieWnd, (hWnd, lp) =>             {                 var text = new StringBuilder(500);                 User32.GetClassName(hWnd, text, text.Capacity);                 if (text.ToString() == "MicrosoftSilverlight")                 {                     frameHWnd = hWnd;                     return false;                 }                 return true;             }, 0);             return frameHWnd;         }     } }

But this is not the end of the story. I needed to pass the captured data to the Silverlight application. I decided that my WinForm app will host a WCF Silverlight friendly service. To do so, I had to deal again with the cross-domain policy file. Thanks to WCF Rest support I could have add a cross-domain policy to a self-hosted service:

  [ServiceContract(Namespace = "http://localhost:8086/")]   public interface ICrossDomainService   {       [OperationContract, WebGet(UriTemplate = "/crossdomain.xml", BodyStyle = WebMessageBodyStyle.Bare)]       Stream GetPolicy();   }    [ServiceBehavior(InstanceContextMode = InstanceContextMode.Single, Namespace = "http://localhost:8086/")]   public class CrossDomainService : ICrossDomainService   {        #region IPolicyRetriever Members        [OperationBehavior]       public Stream GetPolicy()       {            const string result = @"<?xml version=""1.0""?>                <!DOCTYPE cross-domain-policy SYSTEM                      ""http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd"">                <cross-domain-policy>                    <allow-access-from domain=""*"" />                    <allow-http-request-headers-from domain=""*"" headers=""SOAPAction""/>                </cross-domain-policy>";            WebOperationContext.Current.OutgoingResponse.ContentType = "application/xml";            return new MemoryStream(Encoding.UTF8.GetBytes(result));        }