http://forums.microsoft.com/MSDN/ShowPost.aspx?PostID=77027
http://www.gamedev.net/reference/start_here/
http://blogs.msdn.com/danielfe/archive/2006/05/16/599054.aspx - these are GDI games but they will be very easy to convert to XNA and that would be a good learnign excersize.
However, Microsoft did provide a wonderful getting started tutorial. In that tutorial they have laid the groundwork for what could potential become your pong. I would highly recommend that you walk through that tutorial and being playing with your final result.
You can access the tutorial by click on the "Help" menu and selecting "Contents". Now, expand the XNA category, the XNA Game Studio Express category and then the Getting Started with XNA category.
Click on the "Your First XNA Game" help topic. This will display a very basic and step by step instructions for how to get your first XNA game up and running.
It wouldn't take too much work to take that example through to a basic implementation of pong.
For a visual step-by-step of how to access Microsoft's tutorial, I've written a tutorial on how to find Microsoft's "Your First XNA Game" tutorial. Hope that helps.