Pan and tilt camera

I wnat to simulate a pan and tilt camera.
Therefor I need to attach a camera with a joint to a robot frame.
For testing issues I used the Pendulum exampleLink and tried to attach a camera at the pendulum Box itself.

I inserted a cameraEntity in the Pendulum class and added it as children to the pendulum But the camera is not accessable

Code Snippet


private void CreatePendulum(Vector3 pos)
{
BoxShapeProperties prop = new BoxShapeProperties(
"pendulumbox",
mass,
new Pose(new Vector3(0, 0, 0)),
new Vector3(radius, length, radius));
Shape shape = new BoxShape(prop);
_pend = new SingleShapeEntity(shape, base.State.Pose.Position);
_pend.State.Name = "pendulum";
_pend.State.Pose.Position = new Vector3(pos.X + 0f, pos.Y - length / 2 + radius / 2, pos.Z - radius / 2 - length / 2);

// low resolution, wide Field of View
_camera = new CameraEntity(320, 240);
_camera.State.Name = "PendCam";
// just on top of the bot
_camera.State.Pose.Position = pos;
// camera renders in an offline buffer at each frame
// required for service
_camera.IsRealTimeCamera = true;

_pend.Children.Add(_camera);
_camera.Parent = _pend;
}


The second time I tried to call the initialize function of the camera in the Intialize Function of the pendulum entity instead of inserting it as child, but still no cam accessable.

When I add the cam as child in my simulator-class to the pendulum object it is added in the center of the whole object but in a fixed coordinate frame without movement because the complex object itself does not move, only the pendulum.

So what other possibility's do I have to attach a camera to a jointed object or to a joint itself?

[2071 byte] By [MikeR.] at [2008-1-5]
# 1

Hi Mike,

I'm facing the same problem on a humanoid. The camera is static when set as a child of a jointed entity, and refuses to be a child of a link as far as I can tell.

One solution might be to have it be a kinematic entity controlled explicitly.

The Robudog has a controlable pan tilt that seems seperate from the head, so direct control might be the only available solution.

Anyone?

Chris

ChrisKilner at 2007-10-3 > top of Msdn Tech,Microsoft Robotics Studio,Microsoft Robotics - Simulation...
# 2
Thanks for the fast reply.
So I'll try to update the positions of the camera by the entity position. Thats a good idea.
But it would be nice to know if theres a possibility to attach a camera to a joint.

MikeR. at 2007-10-3 > top of Msdn Tech,Microsoft Robotics Studio,Microsoft Robotics - Simulation...
# 3

Cameras currently cannot be attached to joints. In order for a joint to connect two entities, both entities are required to have PhysicsEntities. Since the camera shouldn't really be affected by physics, it doesn't have a PhysicsEntity and cannot be connected by a joint.

What do you mean when you say that the camera is not accessable?

We have been working to add more support for child entities. In our next release, you should be able to add a camera to an entity and it will follow the entity around as if it were glued to it. Additionally, if you add an entity as a child to another entity, and both entities have physics shapes, a rigid joint will be made to connect the two. The joint will then be able to be modified as you like. Please let us know if this is not the type of functionality you're looking for! The idea is to be able to compose entities easily and intuitively inside the simulator.

In the next release, although you will not be able to attach a camera to a joint, you can attach it to another entity. That entity may itself be attached to another entity by a joint, which should hopefully yield your desired effect.

For the time being, however, there is poor support for child entities. I recommend adding entities as fields of other entities rather than adding them to the Children collection, and updating them manually.

Stanlo at 2007-10-3 > top of Msdn Tech,Microsoft Robotics Studio,Microsoft Robotics - Simulation...
# 4
When I add a camera as child it is like "Glued "to the Parent entity.
I want to integrate a camera to a custom build entity derived from visual entity. I have for example two Boxes connected with a joint so that one box can swing when a force is applied.
At the swinging Box a camera should be attached.

Box 1
_ <-- Joint
|
| <- Moveable Box (Pendulum)

When I add the camera to the whole entity as child, it is fixed in the coordinate of the whole Pendulum entity and does not move when a force is applied. Only the pendulum Box moves.

When I define the camera direct in the custom, from visual-entity derived, object and initialize it in the Initialize function:
cam.Initialize(device, physicsEngine)

there is no camera in the simulators camera menu.

The same effect when I insert the cam as child of the Pendulum Box (singleshape entity) in my visual Entity Pendulum.

MikeR. at 2007-10-3 > top of Msdn Tech,Microsoft Robotics Studio,Microsoft Robotics - Simulation...
# 5

Try adding the camera both to the movable box using InsertEntity, and to the simulator using SimulationEngine.GlobalInstancePort.Insert. The camera will appear in the property grid as a separate entity, but its position and rotation should be dependent on the movable box.

Stanlo at 2007-10-3 > top of Msdn Tech,Microsoft Robotics Studio,Microsoft Robotics - Simulation...
# 6
Thank you very much!
That did the job.
The cam is now moving with the pendulum. Now I can try to attach it to a box which is connected with a joint to my robot and control the joint with a service!
MikeR. at 2007-10-3 > top of Msdn Tech,Microsoft Robotics Studio,Microsoft Robotics - Simulation...
# 7
I'm trying to add camera in kuka's arm, can you show us the code?
Irotech at 2007-10-3 > top of Msdn Tech,Microsoft Robotics Studio,Microsoft Robotics - Simulation...
# 8
Sorry for my late answer but I've been on vacation.
So here is the code of the camera attached to the Pendulum:

This is the code for the SimulationService class:

Code Snippet

void AddPendulumWithCam(Vector3 position)
{
//Create the pendulum
PendulumWithCam pendWithCam = new PendulumWithCam(position);
pendWithCam.State.Name = "PendulumWithCam";

//create the camera for the pendulum and insert it separate to the simulation engine
//pose is relative to pendulum coord system --> cam is a child
// Y pose is changed to put it on the end of the pend
pendWithCam.CreateCamera("PendulumCam", new Vector3(0, - 0.5f * position.Y, 0));

// Start simulated webcam service for Pendulums cam
simwebcam.Contract.CreateService(
ConstructorPort,
Microsoft.Robotics.Simulation.Partners.CreateEntityPartner(
"http://localhost/" + pendWithCam.PendulumsCamera.State.Name)
);

// IMPORTANT: even the cam is child of the pendulum entity
// it has to be inserted separate to the physics engine
SimulationEngine.GlobalInstancePort.Insert(pendWithCam.PendulumsCamera);

//Insert whole moveable Cam in Simulation
SimulationEngine.GlobalInstancePort.Insert(pendWithCam);
}

Code Snippet

#region Pendulum With Camera
public class PendulumWithCam : VisualEntity
{
private CameraEntity _camera = null;

private PhysicsJoint _joint = null;
private SingleShapeEntity _support;
private SingleShapeEntity _pend;

/*
* This is the Cam attached to the swinging part
*/
[DataMember]
public CameraEntity PendulumsCamera
{
get { return _camera; }
set { _camera = value; }
}

/*
* Joint connecting the static and moving part
*/
[DataMember]
public Joint Joint
{
get { return _joint; }
set { _joint = (PhysicsJoint)value; }
}

/*
* The static part of the pendulum
*/
[DataMember]
public SingleShapeEntity Support
{
get { return _support; }
set { _support = value; }
}

/*
* The swinging Part
*/
[DataMember]
public SingleShapeEntity Pendulum
{
get { return _pend; }
set { _pend = value; }
}

private const float radius = 0.05f;
private const float length = 0.50f;
private const float mass = 10.0f;

/*
* some basic Constructors
*/
public PendulumWithCam()
{
}

public PendulumWithCam(Vector3 initialPosition)
{
base.State.Pose.Position = initialPosition;
}

/*
* Create the joint and set propertys
*/
private void CreateJoint()
{
JointAngularProperties commonAngular = new JointAngularProperties();
//make twisting possible
commonAngular.TwistMode = JointDOFMode.Free;
_joint = PhysicsJoint.Create(new JointProperties(commonAngular, null, null));
_joint.State.Name = "PendulumJoint";
}

/*
* Create the static Part of the pendulum
* As a simple single Shape entity
*/
private void CreateSupport(Vector3 pos)
{
BoxShapeProperties prop = new BoxShapeProperties(
"supportbox",
mass,
new Pose(new Vector3(0, 0, 0)), //pose of shape in entitys coordframe
new Vector3(radius, radius, length)); //dimensions
Shape shape = new BoxShape(prop);
_support = new SingleShapeEntity(shape,
base.State.Pose.Position); //pose of support-Entity in world coordframe
_support.State.Name = "support";
Console.WriteLine(base.State.Pose.Position);
}

/*
* Create the swinging part
* also as single shape entity
*/
private void CreatePendulum(Vector3 pos)
{
BoxShapeProperties prop = new BoxShapeProperties(
"pendulumbox",
mass,
new Pose(new Vector3(0, 0, 0)), //pose of shape in entitys coordframe
new Vector3(radius, length, radius));
Shape shape = new BoxShape(prop);
_pend = new SingleShapeEntity(shape,
new Vector3(base.State.Pose.Position.X, //same as support
(base.State.Pose.Position.Y - 0.5f * length + 0.5f * radius),
base.State.Pose.Position.Z - 0.5f * length - 0.5f * radius));
//base.State.Pose.Position); //pose of support-Entity in world coordframe
_pend.State.Name = "pendulum";
}

public void CreateCamera(String camID, Vector3 position)
{
// low resolution, wide Field of View
_camera = new CameraEntity(320, 240);

//Set the name of the camera -> for the service
_camera.State.Name = camID;
//set position
_camera.State.Pose.Position = position;

_camera.IsRealTimeCamera = true;

}

public override void Initialize(xnagrfx.GraphicsDevice device, PhysicsEngine physicsEngine)
{
try
{
InitError = string.Empty;

// create joint
if (_joint == null)
{
CreateJoint();
}
else
{
_joint = PhysicsJoint.Create(_joint.State);
}

//create entitys and Cam
if (_support == null) CreateSupport(base.State.Pose.Position);
if (_pend == null) CreatePendulum(base.State.Pose.Position);
if (_camera == null) CreateCamera("PendCam", base.State.Pose.Position);

base.Initialize(device, physicsEngine);
_support.Initialize(device, physicsEngine);
_pend.Initialize(device, physicsEngine);

// Pose of the joint is important because non static objects positions will be forced to the joints attach point
_joint.State.Connectors[0] = new EntityJointConnector(
_support,
new Vector3(0, 1, 0),
new Vector3(0, 0, 1),
new Vector3(0f, 0f, - length / 2)
);
_joint.State.Connectors[1] = new EntityJointConnector(
_pend,
new Vector3(0, 1, 0),
new Vector3(0, 0, 1),
new Vector3(0f, length / 2 - radius / 2 , radius / 2)
);

// IMPORTANT: Insert Cam as child of pendulum -> cam pose relative to pendulum.
_pend.InsertEntity(_camera);

//make _support a fixed object
_support.PhysicsEntity.IsKinematic = true;

PhysicsEngine.InsertJoint(_joint);

Flags |= VisualEntityProperties.DoCompletePhysicsShapeUpdate;
}
catch (Exception ex)
{
HasBeenInitialized = false;
InitError = ex.ToString();
}
}

public override void Update(FrameUpdate update)
{
base.Update(update);
_support.Update(update);
_pend.Update(update);
}

public override void Render(RenderMode renderMode, MatrixTransforms transforms, CameraEntity currentCamera)
{
base.Render(renderMode, transforms, currentCamera);
_support.Render(renderMode, transforms, currentCamera);
_pend.Render(renderMode, transforms, currentCamera);
}

public override void Dispose()
{
base.Dispose();
_support.Dispose();
_pend.Dispose();
}
}
#endregion

In general it is just a custom entity which is derived from visual entity.

This custom one contains 2 Entitys , same as in the pendulum example

One of this entity gets a camera attached as child.

The important part as mentioned above is to insert the cam also in the simulation service to the Engine!
I think that normally the parent entity takes care of this, but not in this case.

Tomorrow I'll try to build a pan and tilt cam

regards

mike

MikeR. at 2007-10-3 > top of Msdn Tech,Microsoft Robotics Studio,Microsoft Robotics - Simulation...

Microsoft Robotics Studio

Site Classified