My XNA and MDX documentation seem to overlap, so I'm not 100% sure if this is accurate.
I think the operation in the RenderState you're referring to is the AlphaBlendOperation, which controls how blending works globally. The alpha arguments you want (and the operation for that matter) used to be set on the device.TextureStates collection in MDX. This collection is no longer available on XNA's GraphicsDevice, which makes sense since it's part of the Fixed Function Pipeline that was dropped. One possible way to do this in XNA would be in a shader/effect, which is probably also the recommended pratice.
Hope this helps :)
Oops. I should have said SourceBlend, and DestinationBlend. The ones with Alpha prefix are for blending into the frame buffer alpha channel (per Shawn Hargreaves in another post). Sorry for the confusion.