Grabbing a Video

Feb 2, 2010 at 8:08 PM

What is your suggestion to add frame capture (ala VideoCaptureElement EnableSampleGrabbing) from a straight video file source?

DShow investigations get very messy without proper clues...

Thank you for you work!

Coordinator
Feb 2, 2010 at 8:10 PM
Try this class:  http://wpfmediakit.codeplex.com/SourceControl/changeset/view/38556#245585
Feb 2, 2010 at 8:23 PM

If i understand correctly with MediaDet i can get video frames given a time position.

But what about getting frames as the video normally plays at his own speed, through a (rendered or not) video renderer?

EX: VMR9 renderless -> d3d texture -> d3dimage

or VMR9 renderless -> sample grabber

 

 

Coordinator
Feb 2, 2010 at 8:28 PM
You will have to use the SampleGrabber and ISampleGrabberCB.  The video capture element has it, but you'd have to copy it over to the MediaUriPlayer.
Feb 2, 2010 at 8:44 PM

15 minutes and it was done.
Never had such a quick success with DShow in years ;)

Thank you for your guidance!

 

Coordinator
Feb 2, 2010 at 8:46 PM
LOL!  Glad it worked for you!  And 15 minutes is a great turn around time!

Thats awesome :)
Feb 15, 2010 at 5:48 PM

Hi, can you post that SampleGrabber code? I too would like the ability to capture the image like a picture.

Mar 2, 2010 at 8:58 AM

Hello,

sorry for the late response, here is the MediaUriPlayer i modified with ISampleGrabber functionality.
It is just a copy/paste/tweak/run code (15mins), so you may have to refine it:

Btw: samplegrabbing is so slow i cannot use this on Atom machines, also faster ones. I wonder if anyone found a solution for grabbing faster (es: rendering directly to an offscreen surface).

 

#region Usings
using System;
using System.Runtime.InteropServices;
using WPFMediaKit.DirectShow.Interop;
using WPFMediaKit.MediaFoundation.Interop.Misc;
using AMMediaType = WPFMediaKit.DirectShow.Interop.AMMediaType;
using VideoInfoHeader = WPFMediaKit.DirectShow.Interop.VideoInfoHeader;
using System.Drawing;
using System.Drawing.Imaging;
#endregion

namespace WPFMediaKit.DirectShow.MediaPlayers
{
    [ComImport, Guid("04FE9017-F873-410E-871E-AB91661A4EF7")]
    internal class FFDShow
    {
          
    }
    /// <summary>
    /// The MediaUriPlayer plays media files from a given Uri.
    /// </summary>
    public class MediaUriPlayer : MediaSeekingPlayer, ISampleGrabberCB
    {
        [DllImport("Kernel32.dll", EntryPoint = "RtlMoveMemory")]
        private static extern void CopyMemory(IntPtr destination, IntPtr source, [MarshalAs(UnmanagedType.U4)] int length);

        /// <summary>
        /// The name of the default audio render.  This is the
        /// same on all versions of windows
        /// </summary>
        private const string DEFAULT_AUDIO_RENDERER_NAME = "Default DirectSound Device";

        /// <summary>
        /// Set the default audio renderer property backing
        /// </summary>
        private string m_audioRenderer = DEFAULT_AUDIO_RENDERER_NAME;

#if DEBUG
        /// <summary>
        /// Used to view the graph in graphedit
        /// </summary>
        private DsROTEntry m_dsRotEntry;
#endif

        /// <summary>
        /// The DirectShow graph interface.  In this example
        /// We keep reference to this so we can dispose 
        /// of it later.
        /// </summary>
        private IGraphBuilder m_graph;

        /// <summary>
        /// The media Uri
        /// </summary>
        private Uri m_sourceUri;

        /// <summary>
        /// The sample grabber interface used for getting samples in a callback
        /// </summary>
        private ISampleGrabber m_sampleGrabber;

        /// <summary>
        /// Gets or sets if the instance fires an event for each of the samples
        /// </summary>
        public bool EnableSampleGrabbing { get; set; }

        /// <summary>
        /// Fires when a new video sample is ready
        /// </summary>
        public event EventHandler<VideoSampleArgs> NewVideoSample;

        private void InvokeNewVideoSample(VideoSampleArgs e)
        {
            EventHandler<VideoSampleArgs> sample = NewVideoSample;
            if (sample != null) sample(this, e);
        }

        /// <summary>
        /// Gets or sets the Uri source of the media
        /// </summary>
        public Uri Source
        {
            get
            {
                VerifyAccess();
                return m_sourceUri;
            }
            set
            {
                VerifyAccess();
                m_sourceUri = value;
                
                OpenSource();
            }
        }

        /// <summary>
        /// The renderer type to use when
        /// rendering video
        /// </summary>
        public VideoRendererType VideoRenderer
        {
            get;set;
        }

        /// <summary>
        /// The name of the audio renderer device
        /// </summary>
        public string AudioRenderer
        {
            get
            {
                VerifyAccess();
                return m_audioRenderer;
            }
            set
            {
                VerifyAccess();

                if(string.IsNullOrEmpty(value))
                {
                    value = DEFAULT_AUDIO_RENDERER_NAME;
                }

                m_audioRenderer = value;
            }
        }

        /// <summary>
        /// Gets or sets if the media should play in loop
        /// or if it should just stop when the media is complete
        /// </summary>
        public bool Loop { get; set; }

        /// <summary>
        /// Is ran everytime a new media event occurs on the graph
        /// </summary>
        /// <param name="code">The Event code that occured</param>
        /// <param name="lparam1">The first event parameter sent by the graph</param>
        /// <param name="lparam2">The second event parameter sent by the graph</param>
        protected override void OnMediaEvent(EventCode code, IntPtr lparam1, IntPtr lparam2)
        {
            if(Loop)
            {
                switch(code)
                {
                    case EventCode.Complete:
                        MediaPosition = 0;
                        break;
                }
            }
            else
                /* Only run the base when we don't loop
                 * otherwise the default behavior is to
                 * fire a media ended event */
                base.OnMediaEvent(code, lparam1, lparam2);
        }

        /// <summary>
        /// Opens the media by initializing the DirectShow graph
        /// </summary>
        protected virtual void OpenSource()
        {
            /* Make sure we clean up any remaining mess */
            FreeResources();

            if (m_sourceUri == null)
                return;

            string fileSource = m_sourceUri.OriginalString;

            if (string.IsNullOrEmpty(fileSource))
                return;

            try
            {
                /* Creates the GraphBuilder COM object */
                m_graph = new FilterGraphNoThread() as IGraphBuilder;

                if (m_graph == null)
                    throw new Exception("Could not create a graph");

                /* Add our prefered audio renderer */
                InsertAudioRenderer(AudioRenderer);

                IBaseFilter renderer = CreateVideoRenderer(VideoRenderer, m_graph, 2);

                var filterGraph = m_graph as IFilterGraph2;

                if (filterGraph == null)
                    throw new Exception("Could not QueryInterface for the IFilterGraph2");

                IBaseFilter sourceFilter;

                /* Have DirectShow find the correct source filter for the Uri */
                int hr = filterGraph.AddSourceFilter(fileSource, fileSource, out sourceFilter);
                DsError.ThrowExceptionForHR(hr);

                var graphBuilder = (ICaptureGraphBuilder2)new CaptureGraphBuilder2();
                /* Set our filter graph to the capture graph */
                hr = graphBuilder.SetFiltergraph(m_graph);
                DsError.ThrowExceptionForHR(hr);

                /* We will want to enum all the pins on the source filter */
                IEnumPins pinEnum;

                hr = sourceFilter.EnumPins(out pinEnum);
                DsError.ThrowExceptionForHR(hr);

                IntPtr fetched = IntPtr.Zero;
                IPin[] pins = { null };

                /* Counter for how many pins successfully rendered */
                int pinsRendered = 0;

                if (VideoRenderer == VideoRendererType.VideoMixingRenderer9)
                {
                    var mixer = renderer as IVMRMixerControl9;

                    if (mixer != null )
                    {
                        VMR9MixerPrefs dwPrefs;
                        mixer.GetMixingPrefs(out dwPrefs);
                        dwPrefs &= ~VMR9MixerPrefs.RenderTargetMask;
                        dwPrefs |= VMR9MixerPrefs.RenderTargetRGB;
                        //mixer.SetMixingPrefs(dwPrefs);
                    }
                }

                /* Test using FFDShow Video Decoder Filter
                var ffdshow = new FFDShow() as IBaseFilter;

                if (ffdshow != null)
                    m_graph.AddFilter(ffdshow, "ffdshow");
                */

                //if (EnableSampleGrabbing)
                {
                    m_sampleGrabber = (ISampleGrabber)new SampleGrabber();
                    SetupSampleGrabber(m_sampleGrabber);
                    hr = m_graph.AddFilter(m_sampleGrabber as IBaseFilter, "SampleGrabber");
                    DsError.ThrowExceptionForHR(hr);
                }

                /* Loop over each pin of the source filter */
                while (pinEnum.Next(pins.Length, pins, fetched) == 0)
                {
                    if (filterGraph.RenderEx(pins[0],
                                             AMRenderExFlags.RenderToExistingRenderers,
                                             IntPtr.Zero) >= 0)
                        pinsRendered++;

                    Marshal.ReleaseComObject(pins[0]);
                }

                Marshal.ReleaseComObject(pinEnum);
                Marshal.ReleaseComObject(sourceFilter);

                if (pinsRendered == 0)
                    throw new Exception("Could not render any streams from the source Uri");

#if DEBUG
                /* Adds the GB to the ROT so we can view
                 * it in graphedit */
                m_dsRotEntry = new DsROTEntry(m_graph);
#endif
                /* Configure the graph in the base class */
                SetupFilterGraph(m_graph);

                HasVideo = true;
                /* Sets the NaturalVideoWidth/Height */
                //SetNativePixelSizes(renderer);
            }
            catch (Exception ex)
            {
                /* This exection will happen usually if the media does
                 * not exist or could not open due to not having the
                 * proper filters installed */
                FreeResources();

                /* Fire our failed event */
                InvokeMediaFailed(new MediaFailedEventArgs(ex.Message, ex));
            }
           
            InvokeMediaOpened();
        }

        /// <summary>
        /// Inserts the audio renderer by the name of
        /// the audio renderer that is passed
        /// </summary>
        protected virtual void InsertAudioRenderer(string audioDeviceName)
        {
            if(m_graph == null)
                return;

            AddFilterByName(m_graph, FilterCategory.AudioRendererCategory, audioDeviceName);
        }

        /// <summary>
        /// Frees all unmanaged memory and resets the object back
        /// to its initial state
        /// </summary>
        protected override void FreeResources()
        {
#if DEBUG
            /* Remove us from the ROT */
            if (m_dsRotEntry != null)
            {
                m_dsRotEntry.Dispose();
                m_dsRotEntry = null;
            }
#endif

            /* We run the StopInternal() to avoid any 
             * Dispatcher VeryifyAccess() issues because
             * this may be called from the GC */
            StopInternal();

            /* Let's clean up the base 
             * class's stuff first */
            base.FreeResources();

            if(m_graph != null)
            {
                Marshal.ReleaseComObject(m_graph);
                m_graph = null;

                /* Only run the media closed if we have an
                 * initialized filter graph */
                InvokeMediaClosed(new EventArgs());
            }
        }

        /// <summary>
        /// Sets the capture parameters for the video capture device
        /// </summary>
        private bool SetVideoCaptureParameters(ICaptureGraphBuilder2 capGraph, IBaseFilter captureFilter, Guid mediaSubType)
        {
            /* The stream config interface */
            object streamConfig;

            /* Get the stream's configuration interface */
            int hr = capGraph.FindInterface(PinCategory.Capture,
                                            MediaType.Video,
                                            captureFilter,
                                            typeof(IAMStreamConfig).GUID,
                                            out streamConfig);

            DsError.ThrowExceptionForHR(hr);

            var videoStreamConfig = streamConfig as IAMStreamConfig;

            /* If QueryInterface fails... */
            if (videoStreamConfig == null)
            {
                throw new Exception("Failed to get IAMStreamConfig");
            }

            /* The media type of the video */
            AMMediaType media;

            /* Get the AMMediaType for the video out pin */
            hr = videoStreamConfig.GetFormat(out media);
            DsError.ThrowExceptionForHR(hr);

            /* Make the VIDEOINFOHEADER 'readable' */
            var videoInfo = new VideoInfoHeader();
            Marshal.PtrToStructure(media.formatPtr, videoInfo);

            int FPS = 30, DesiredWidth = 320, DesiredHeight = 240;

            /* Setup the VIDEOINFOHEADER with the parameters we want */
            videoInfo.AvgTimePerFrame = DSHOW_ONE_SECOND_UNIT / FPS;
            videoInfo.BmiHeader.Width = DesiredWidth;
            videoInfo.BmiHeader.Height = DesiredHeight;

            if (mediaSubType != Guid.Empty)
            {
                videoInfo.BmiHeader.Compression = new FourCC(mediaSubType).ToInt32();
                media.subType = mediaSubType;
            }

            /* Copy the data back to unmanaged memory */
            Marshal.StructureToPtr(videoInfo, media.formatPtr, false);

            /* Set the format */
            hr = videoStreamConfig.SetFormat(media);

            /* We don't want any memory leaks, do we? */
            DsUtils.FreeAMMediaType(media);

            if (hr < 0)
                return false;

            return true;
        }

        private void SetupSampleGrabber(ISampleGrabber sampleGrabber)
        {
            var mediaType = new AMMediaType
            {
                majorType = MediaType.Video,
                subType = MediaSubType.RGB24,
                formatType = FormatType.VideoInfo
            };

            int hr = sampleGrabber.SetMediaType(mediaType);

            DsUtils.FreeAMMediaType(mediaType);
            DsError.ThrowExceptionForHR(hr);

            hr = sampleGrabber.SetCallback(this, 0);
            DsError.ThrowExceptionForHR(hr);
        }

        private System.Drawing.Bitmap m_videoFrame;

        private void InitializeBitmapFrame(int width, int height)
        {
            if (m_videoFrame != null)
            {
                m_videoFrame.Dispose();
            }

            m_videoFrame = new Bitmap(width, height, PixelFormat.Format24bppRgb);
        }


        int ISampleGrabberCB.SampleCB(double sampleTime, IMediaSample pSample)
        {
            var mediaType = new AMMediaType();

            /* We query for the media type the sample grabber is using */
            int hr = m_sampleGrabber.GetConnectedMediaType(mediaType);

            var videoInfo = new VideoInfoHeader();

            /* 'Cast' the pointer to our managed struct */
            Marshal.PtrToStructure(mediaType.formatPtr, videoInfo);

            /* The stride is "How many bytes across for each pixel line (0 to width)" */
            int stride = Math.Abs(videoInfo.BmiHeader.Width * (videoInfo.BmiHeader.BitCount / 8 /* eight bits per byte */));
            int width = videoInfo.BmiHeader.Width;
            int height = videoInfo.BmiHeader.Height;

            if (m_videoFrame == null)
                InitializeBitmapFrame(width, height);

            if (m_videoFrame == null)
                return 0;

            BitmapData bmpData = m_videoFrame.LockBits(new Rectangle(0, 0, width, height),
                                                       ImageLockMode.ReadWrite,
                                                       PixelFormat.Format24bppRgb);

            /* Get the pointer to the pixels */
            IntPtr pBmp = bmpData.Scan0;

            IntPtr samplePtr;

            /* Get the native pointer to the sample */
            pSample.GetPointer(out samplePtr);

            int pSize = stride * height;

            /* Copy the memory from the sample pointer to our bitmap pixel pointer */
            CopyMemory(pBmp, samplePtr, pSize);

            m_videoFrame.UnlockBits(bmpData);

            InvokeNewVideoSample(new VideoSampleArgs { VideoFrame = m_videoFrame });

            DsUtils.FreeAMMediaType(mediaType);

            /* Dereference the sample COM object */
            Marshal.ReleaseComObject(pSample);
            return 0;
        }

        int ISampleGrabberCB.BufferCB(double sampleTime, IntPtr pBuffer, int bufferLen)
        {
            throw new NotImplementedException();
        }
    }
}