Really smooth playback

Dec 28, 2009 at 2:02 PM

Hi!

I have rewrite my TV capture program from C++ to WPF using D3DImage and EVR Custom Presenter and I am very disappointment about playback smoothness. The problem is of course the need to call D3DImage methods on the GUI Thread. Any delay on the GUI thread is very visible (and sometimes unavoidable like creating/destroying UI controls). I was wandering if there is a method to not give up on smoothness and still use WPF to render OSD over video control and I think I've found a solution:

It should be possible to render video directly on the window (with EVR without Custom Presenter) and mix it (with EVR's mixer) with WPF UI rendered on Direct3D surface. Reversed situation. In that case we don't need to synchronize to WPF UI thread, so at least video will achieve maximum smoothness. The WPF UI may be slower this time, but for me it's much better that way. And I think it's a way to use less CPU. So thanks for your post:

"How to get access to WPF’s internal Direct3D guts"

because I'm going to try!! This is the proof it is possible too, I think:

http://www.youtube.com/watch?v=OJgHMA2uJO0

Marek

 

Dec 28, 2009 at 3:23 PM

Sorry for bothering, but I've found best solution (technically you found :)) to smoothness playback + all WPF power + small CPU footprint.

Your previous approach - hacking direct show graph in WPF's MediaElement is the best! It cannot use internally D3DImage, because it has much better performance.

The only drawback is... it's a hack and may not work in the future, but for me the difference between MediaElement and D3DImage is very significant - completely disqualifing D3DImage.

Thanks for being there first and showing me the way. Every path I try, you were there before :)

Marek

Dec 28, 2009 at 4:27 PM
Marek,

I'm glad you found a solution...and I'm glad it was my code that was able to help!

There are some performance issues with D3DImage, most notably the ones I've encountered are: XP, non IDirect3DDevice9Ex usage.  Other than that, it is the fastest way to do raster updates using supported APIs.

I have not experienced performance issues as you've described with D3DImage (I also made a TV Tuner control).  I generally get about 10% higher CPU usage than MediaElement (on a single core scale).  Also, I'm usually very careful to never do anything I don't have to on the UI thread (I only do, D3DImage::AddDirtyRect and start a thread to run on the DShow components).  Do you experience these issues also if you use my WPF MediaKit.  If so, maybe the move over to the MediaElement hack I made is the best choice.

I'm not sure if this will help or not, but I have some D3DImage extension methods to do (most) of the operations on your own thread.  Here it is:

public static class D3DImageExtensions

    {

        private static FieldInfo m_lockCountField;

        private static FieldInfo m_isDirtyField;

        private static MethodInfo m_subscribeToCommittingBatchMethod;

        private static FieldInfo m_isChangePendingField;

        private static MethodInfo m_writePostScriptMethod;

        private static MethodInfo m_lockImplMethodInfo;

        private static FieldInfo m_pInteropDeviceBitmapField;

 

        [DllImport("wpfgfx_v0300.dll", EntryPoint = "InteropDeviceBitmap_AddDirtyRect")]

        private static extern int AddDirtyRectNative(int x, int y, int w, int h, SafeHandlepInteropDeviceBitmap);

 

        static D3DImageExtensions()

        {

            var t = typeof(D3DImage);

            m_lockCountField = t.GetField("_lockCount"BindingFlags.NonPublic |BindingFlags.Instance);

            m_isDirtyField = t.GetField("_isDirty"BindingFlags.NonPublic |BindingFlags.Instance);

           m_subscribeToCommittingBatchMethod = t.GetMethod("SubscribeToCommittingBatch",BindingFlags.NonPublic | BindingFlags.Instance);

            m_isChangePendingField = t.GetField("_isChangePending"BindingFlags.NonPublic | BindingFlags.Instance);

            m_writePostScriptMethod = t.GetMethod("WritePostscript"BindingFlags.NonPublic | BindingFlags.Instance);

            m_lockImplMethodInfo = t.GetMethod("LockImpl"BindingFlags.NonPublic |BindingFlags.Instance);

            m_pInteropDeviceBitmapField = t.GetField("_pInteropDeviceBitmap",BindingFlags.NonPublic | BindingFlags.Instance);

        }

 

        public static void LockEx(this D3DImage d3dImage)

        {

            m_lockImplMethodInfo.Invoke(d3dImagenew object[] { Duration.Forever });

        }

 

        public static void UnlockEx(this D3DImage d3dImage)

        {

            uint lockCount = (uint)m_lockCountField.GetValue(d3dImage);

            bool isDirty = (bool)m_isDirtyField.GetValue(d3dImage);

           

            lockCount--;

 

            m_lockCountField.SetValue(d3dImage, lockCount);

 

            if(isDirty && (lockCount == 0))

            {

                m_subscribeToCommittingBatchMethod.Invoke(d3dImagenull);

            }

 

            bool isChangePending = (bool)m_isChangePendingField.GetValue(d3dImage);

           

            if(isChangePending)

            {

                isChangePending = false;

                m_isChangePendingField.SetValue(d3dImage, isChangePending);

 

                d3dImage.Dispatcher.Invoke((Action)delegate

                                                       {

                                                           m_writePostScriptMethod.Invoke(d3dImagenull);

                                                       });

              

            }

        }

 

        public static void AddDirtyRectEx(this D3DImage d3dImageInt32Rect dirtyRect)

        {

            var pInteropDeviceBitmap = (SafeHandle)m_pInteropDeviceBitmapField.GetValue(d3dImage);

 

            int hr = AddDirtyRectNative(dirtyRect.X, dirtyRect.Y, dirtyRect.Width, dirtyRect.Height, pInteropDeviceBitmap);

 

            m_isChangePendingField.SetValue(d3dImagetrue);

            m_isDirtyField.SetValue(d3dImagetrue);

        }

    }


On Mon, Dec 28, 2009 at 7:23 AM, pirat_komputerowy <notifications@codeplex.com> wrote:

From: pirat_komputerowy

Sorry for bothering, but I've found best solution (technically you found :)) to smoothness playback + all WPF power + small CPU footprint.

Your previous approach - hacking direct show graph in WPF's MediaElement is the best! It cannot use internally D3DImage, because it has much better performance.

The only drawback is... it's a hack and may not work in the future, but for me the difference between MediaElement and D3DImage is very significant - completely disqualifing D3DImage.

Thanks for being there first and showing me the way. Every path I try, you were there before :)

Marek

Read the full discussion online.

To add a post to this discussion, reply to this email (WPFMediaKit@discussions.codeplex.com)

To start a new discussion for this project, email WPFMediaKit@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com


Dec 28, 2009 at 6:17 PM

Hi Jeremiah!

Thanks for your quick replay! You're real hacker, I see :-))). Your D3DImageExtensions class of course works (I've tried it), but it has the same main problem:

- Dispatcher.Invoke() for every frame (25 times per second)

When GUI Thread is busy, you will surely see stop-frames. You loose one of the most EVR's benefits - perfect timing. I don't care as much for the CPU usage. The difference is even smaller for me (~4% CPU). It's acceptable. But I can see stop-frames when I use other programs. Even other programs (like a card game - SolSuite2004 for example) can slow down animation (I don't understand it, maybe it is because of additional bitmap copying from backbuffer to front buffer in D3DImage, which uses more power of GPU, I don't know, CPU level is ok). This thing never happens with original MediaElement. It's not really bad, though, but If I can have it better, I will try that better (for me) solution. It's because I saw a difference. If I didn't I wouldn't care of more CPU usage, but I don't like worse smoothness and there is a little impact on other graphically intensive applications (I don't know from).

Anyway, as I said it's you who showed me both paths, so thanks again!!!! I like the first better :)) Without you it wouldn't be possible for me to test it so fast!!

Marek

P.S. Oh. I've just read that I will have a communication problem with DirectShow thread after initialization :) Microsoft doesn't make it easy! But I hope I can resolve it by creating my window (hwnd) in initialization phase and later use it (send it messages on demand) for callbacks.

 

Dec 28, 2009 at 6:27 PM
I think you are on the right track for communicating with the DShow stuff.  In my mediakit, I have a separate thread and dispatcher running the dshow stuff.  I did this because WPF uses STA and DShow has blocking calls.  I used to have it STA, but found MTA was easier in the case that the thread calling the allocator is yet another thread!

There are some issues with the media element hack, but none I can see getting in the way with an analog tuner.  Basically, the WPF EVR presenter ignores dynamic pin changes.  So if you did an ATSC/DVB/HD tuner, the resolution changes with different broadcast channels.  This doesn't happen on analog as you only get one uncompressed resolution.

Good luck man!  Don't be shy about hitting me up if you hit a roadblock!

-Jer
On Mon, Dec 28, 2009 at 10:17 AM, pirat_komputerowy <notifications@codeplex.com> wrote:

From: pirat_komputerowy

Hi Jeremiah!

Thanks for your quick replay! You're real hacker, I see :-))). Your D3DImageExtensions class of course works (I've tried it), but it has the same main problem:

- Dispatcher.Invoke() for every frame (25 times per second)

When GUI Thread is busy, you will surely see stop-frames. You loose one of the most EVR's benefits - perfect timing. I don't care as much for the CPU usage. The difference is even smaller for me (~4% CPU). It's acceptable. But I can see stop-frames when I use other programs. Even other programs (like a card game - SolSuite2004 for example) can slow down animation (I don't understand it, maybe it is because of additional bitmap copying from backbuffer to front buffer in D3DImage, which uses more power of GPU, I don't know, CPU level is ok). This thing never happens with original MediaElement. It's not really bad, though, but If I can have it better, I will try that better (for me) solution. It's because I saw a difference. If I didn't I wouldn't care of more CPU usage, but I don't like worse smoothness and there is a little impact on other graphically intensive applications (I don't know from).

Anyway, as I said it's you who showed me both paths, so thanks again!!!! I like the first better :)) Without you it wouldn't be possible for me to test it so fast!!

Marek

P.S. Oh. I've just read that I will have a communication problem with DirectShow thread after initialization :) Microsoft doesn't make it easy! But I hope I can resolve it by creating my window (hwnd) in initialization phase and later use it (send it messages on demand) for callbacks.

 

Read the full discussion online.

To add a post to this discussion, reply to this email (WPFMediaKit@discussions.codeplex.com)

To start a new discussion for this project, email WPFMediaKit@discussions.codeplex.com

You are receiving this email because you subscribed to this discussion on CodePlex. You can unsubscribe or change your settings on codePlex.com.

Please note: Images and attachments will be removed from emails. Any posts to this discussion will also be available online at codeplex.com