Video/Animation Playback

Hi,

I'm currently writing a library (for XNA/DirectX) for

animation/soundless video playback, and I have a few design questions

for the pros out there if you don't mind answering them:

  1. I'm having my animation class inherit from game component, and

    update the frame depending on game time in the overriden Update method,

    and render it in the overidden Draw method, but even when I preload my

    textures to render, playback is choppy. Not sure exactly why, but I

    notice this only when I have significant number of frames (I noticed it

    with 100 frames, but *not* with 10/15). This problem may be linked and

    solved by the 2nd point, so please read on.

  2. When I preload all the textures for the frames in the animation

    (and the no. of frames is small, as explained in point 1), the

    animation plays back perfectly. Now I've also added the capability

    for future frame textures to be loaded on-the-fly asynchronously (using

    delegates) whilst the animation is playing, in order to a) I can't

    preload all the textures for a big video since they can have 1000s of

    frames... ba) To minimize loading time. From this I've discovered

    that the problem is caused by the slow loading time of frame textures

    in comparison to the playback rate (typically about 300ms compared to

    30ms). Clearly the playback will overtake the on-the-fly preloading

    rather quickly, causing playback to wait up, making it appear choppy.

    So all I want to do it somehow quicken the preloading time for

    textures. This is my current code:

' Create new frame texture and store it in the array.
Dim creationParams As New TextureCreationParameters( _
0, 0, 0, 0, SurfaceFormat.Unknown, ResourceUsage.None, ResourcePool.Managed, _
Nothing, FilterOptions.None, FilterOptions.None)

_frameTextures(frame) = Texture2D.FromFile(graphicsDevice, _dataStream, byteLength, creationParams)

As you can see I'm loading from a stream (FileStream in my tests).

I've fiddled around with the parameters quite a bit but this code makes

it the quickest - still not quick enough.

I hope that some of you may have ideas on how video rendering is

done by other programs, or more likely just some new ideas. Thanks in

advance for any help!

Alex

P.S. If there are any premade animation playback libraries for XNA

already please do let me know, but I've searched and I doubt it anyway

since XNA is still very young.

[2688 byte] By [Alex-MyRpg] at [2007-12-28]
# 1
Moved to XNA Framework forum
DavidWeller-MSFT at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 2
Creating textures for each frame isn't probably the most efficient way, for two reasons:

  • Creating and destroying GPU resources is pretty slow. This is the

    kind of thing you really want to be doing just once in an initialize

    function, rather than inside your rendering loop.

  • Loading textures using FromFile means you aren't going to be

    getting much in the way of data compression. This may be fine if your

    animation is very low resolution, but for decent quality video,

    uncompressed format is going to take so much space, the hard drive may

    well not be able to keep up with the transfer rate this is requiring!

To fix the first problem, a typical approach would be to create

probably three textures using the Dynamic usage flag in your initialize

function. Then you cycle between them. Each time you want to render a

frame, pick the next texture from your list of three, call SetData on

it to fill it with a new frame of image data, then use it to render,

and rotate the list so you get a different texture on the next frame.

The reason to have three textures rather than just one is that the GPU

renders asynchronously with the CPU, so if you called SetData

immediately the next frame on the same texture you just used, the CPU

would have to stall until the GPU was finished rendering out of that

texture. Using three in rotation gives the GPU time to have finished

rendering the previous frames before the CPU tries to stuff new data

into each texture.

To reduce the amount of bandwidth you are using (assuming this turns

out to be a problem), you will need to do some kind of video

compression. That's not an area I know much about though!

ShawnHargreaves at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 3
Thanks very much for your reply. I think you've pretty much covered exactly how to do it. I'll have to check out exactly how to use SetData (do I use it on the Texture directly or on it's surface?) An example would be great, but you've been very helpful so far. At the moment I'm loading from JPEG images in a screen, which is good for compression (same size as an MPEG), but not good enough for loading speed.
I'll update here on my progress - and oh, would anyone (including you) perhaps be interested in this Video (i.e. Animation) component for XNA when it's complete? Who knows, I may even add sound later
if people are interested.
Alex-MyRpg at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 4
Is anyone interested then? I've very nearly completed the class thanks to Shawn's help, so I may post it up if there is any interest...
Alex-MyRpg at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 5

I'd certainly be interested.

What format do you use for loading the animations? Is it proprietary or a regular format like mpeg, or wmv?

Cheers


Neil

thedo at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 6

I am interested to sounds real nice !

Bill

billspinhoven at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 7
Great, nice to hear people interested. Currently I use my own custom format and I have a seperate converter from AVI > my format. In the future I may adapt it to handle AVIs directly, or possibly even MPEGs, but the next step is to get sound and that's going to be tricky, since there is no equivalent of DirectSound (or none that has the same level of control) in XNA :S
Still, my converter handles compressed AVIs, which are really the same size as MPEGs.
Anyway I've tidied up the source a bit and I just have a few changes to make to it, and the converter before I release both. I shall be posting a download link here any time in the next day or so hopefully. :)

Alex

P.S. Not sure how much medium-term interested this will generate but I'm happy for people to suggest or make changes to the component so I can incorperate them into subsequent releases.

Alex-MyRpg at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 8
I too would be EXTREEEEEEEEEEMELY interested in this functionality as i'm currently trying to get this working myself.
CaptainTenneal at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 9
I'm definitely interested in video playback as well. I'm especially interested in accessing video from a webcam (on PC... not worried about Xbox 360 development right now).
a.d.m at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 10
Good to hear some interest. Now it's not ready *quite* yet, but would you maybe like me to tidy up it all up so you have a drag & drop AVI > my format converter? Then I can just post the code for the class to use in your game. :)
Alex-MyRpg at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 11
Alex-MyRpg wrote:
Good to hear some interest. Now it's not ready *quite* yet, but would you maybe like me to tidy up it all up so you have a drag & drop AVI > my format converter? Then I can just post the code for the class to use in your game. :)

Yes please!

errolian at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 12
Looks very interesting.Why dont you make an online page for this project so others can help in working on this codec. Anyways keep us informed!
Keep up the good work!
Remmie at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 13
Sure, perhaps I'll start up a SourceForge.net project for helper XNA classes, mainly this one at first. :)-
Alex-MyRpg at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...
# 14
I was just thinking of using some video in a game and would greatly appreciate using your class.
NoNameHere at 2007-9-4 > top of Msdn Tech,Game Technologies: DirectX, XNA, XACT, etc.,XNA Framework...