Harmonic And Jazzy Tutorial
Light and Color - Wavefront
Introduction and explanation
What software are we using and why
All of these instructions have been tested using the current version of Ubuntu: Noble Numbat.We are going to use two packages: Gazebo for the physics based Simulator and ROS2 (Robot Operating System) for the control system. There are many releases of each of these, and only some combinations are mutually compatible. Also, since we are focusing on a stable development platform, we will use the Long Term Support (LTS) versions that are current at the time we are posting this. As of 29Aug2024, this means we will use Gazebo Harmonic which is supported until 2028.
Introduction to OBJ and MTL Files (Wavefront)
What good is a model without light and color? They work, but they are seriously boring. One way to add color to a model in a Gazebo Simuation is to add OBJ and MTL files to your model in a sdf file. User note, this is done with Gazebo Harmonic. In the first example, we will create a cube with an object file and add color to it with a material file. In the second example, we will add a texture to the material file and add that texture to the colored cube.
First, a little history. The MTL/OBJ file format was developed by Wavefront Technologies in 1990. The format was openly published 1995 in as manual for Wavefront’s Advanced Visualization Software. The format is described in detail in The Library of Congress Digital Format page (2020, Library of Congress) and on Paul Bourkes OBL (Bourke1), and MTL pages (Bourke2). When Gazebo migrated from Gazebo Classic to Ignition, they stopped supporting Ogre scripts. However, they do support Collada and OBJ+MTL files (Gazebo1). Not all of the functionality in the original Wavefront site format is supported by the Gazebo simulation. For example the ability to reflect the surroundings from objects is not supported at this time. Gazebo Harmonic can reflect light sources, but not objects.
The file structure is shown graphically below:
First, a little history. The MTL/OBJ file format was developed by Wavefront Technologies in 1990. The format was openly published 1995 in as manual for Wavefront’s Advanced Visualization Software. The format is described in detail in The Library of Congress Digital Format page (2020, Library of Congress) and on Paul Bourkes OBL (Bourke1), and MTL pages (Bourke2). When Gazebo migrated from Gazebo Classic to Ignition, they stopped supporting Ogre scripts. However, they do support Collada and OBJ+MTL files (Gazebo1). Not all of the functionality in the original Wavefront site format is supported by the Gazebo simulation. For example the ability to reflect the surroundings from objects is not supported at this time. Gazebo Harmonic can reflect light sources, but not objects.
The file structure is shown graphically below:
Looking at the parts
From here we will look at the four types of files that are needed: the .sdf, the .obf, the .mtl, and the .png
The sdf file is the safe file we have been working with in the previos tutorials - it sets up the simulation, adds a background light and includes the models. In the previous tutorials, we completely detailed the models directly in the sdf file. That is going to change. We will specifiy the model, but the details will be handled in external files.
The object file (.obj) defines the shape and properties of the model and its parts.
The materials file specifies the visual properties of the object. This can include single colors that can be 'painted' on the object.
The final file is a .png image that can contain colpex visual properties of an object - things like woodgrain, or stippling, or an image captured from a camera, or just about anything.
We will explore these in detail, beginning from the bottom the texture file.
Texture files
We will use these in the mtl file next.
Complete Materials File
The material (MTL) file, it contains a definition for each color that can be used by the object file. It defines 10 specific colors that can be used to paint surfaces of objects.
There are two kinds of materials one that has specific color values coded into it, and
The second kind which refers to an external image - the textures we defined above.
We are going to focus on the textures.
Texture definition:
Each material definition starts with a name and then specifies the RGB color for three different lighting conditions
Ambient light, based on the background illumination
Diffuse light, based on direct illumination
Specular light, which controls how reflected light acts.
So these define the general light response that will be applied to the image.
newmtl blue-textureKa 1.0 1.0 1.0Kd 1.0 1.0 1.0Ks 1.0 1.0 1.0map_Ka blue.pngmap_Kd blue.pngmap_Ks blue.png
In the general case:A name of the color definition - newmtl colorName (blue-texture)The value of the ambient color - Ka r b gThe value of the diffuse color - Kd r b gThe value of the specular color -Ks r b gIn our case all three values are 1.0, so the light will uniformly illuminate the image we paint on the surface
Next, the materials fdefinition contains a reference to the texture file, mapped to ambient, diffuse, and specular. map_Ka texture.pngmap_Kd texture.pngmap_Ks texture.png With these defined for each texture, we are ready to put them to work. In the next example, we will show how to subset a texture file, but for now we will be using one texture file per surface.
For this example we have three texture files and three textures
newmtl blue-textureKa 1.0 1.0 1.0Kd 1.0 1.0 1.0Ks 1.0 1.0 1.0map_Ka blue.pngmap_Kd blue.pngmap_Ks blue.png
newmtl red-textureKa 1.0 1.0 1.0Kd 1.0 1.0 1.0Ks 1.0 1.0 1.0map_Ka red.pngmap_Kd red.pngmap_Ks red.png
newmtl green-textureKa 1.0 1.0 1.0Kd 1.0 1.0 1.0Ks 1.0 1.0 1.0map_Ka green.pngmap_Kd green.pngmap_Ks green.png Where to put the png files? In versions previous to Harmonic, the system looked in a hardcoded /meshes/ directory, with Harmonic, they can be anywhere. Next, we will look at the object file. This file contains all of the information needed to build the object. Below is a picture describing the axis, red is x, green is y, and blue is z.
Next, the materials fdefinition contains a reference to the texture file, mapped to ambient, diffuse, and specular. map_Ka texture.pngmap_Kd texture.pngmap_Ks texture.png With these defined for each texture, we are ready to put them to work. In the next example, we will show how to subset a texture file, but for now we will be using one texture file per surface.
For this example we have three texture files and three textures
newmtl blue-textureKa 1.0 1.0 1.0Kd 1.0 1.0 1.0Ks 1.0 1.0 1.0map_Ka blue.pngmap_Kd blue.pngmap_Ks blue.png
newmtl red-textureKa 1.0 1.0 1.0Kd 1.0 1.0 1.0Ks 1.0 1.0 1.0map_Ka red.pngmap_Kd red.pngmap_Ks red.png
newmtl green-textureKa 1.0 1.0 1.0Kd 1.0 1.0 1.0Ks 1.0 1.0 1.0map_Ka green.pngmap_Kd green.pngmap_Ks green.png Where to put the png files? In versions previous to Harmonic, the system looked in a hardcoded /meshes/ directory, with Harmonic, they can be anywhere. Next, we will look at the object file. This file contains all of the information needed to build the object. Below is a picture describing the axis, red is x, green is y, and blue is z.
Defining the Object
Here is a complete copy of the final object file ManyColoredCube.obj
The object file starts by defining the material file that will be used.
mtllib model.mtl
The next section is used to define the vertices of the object. For a cube this would be
v 1.0 1.0 1.0v 0.0 1.0 1.0v 0.0 0.0 1.0v 1.0 0.0 1.0v 1.0 1.0 0.0v 0.0 1.0 0.0v 0.0 0.0 0.0v 1.0 0.0 0.0
This results in the model shown below.
mtllib model.mtl
The next section is used to define the vertices of the object. For a cube this would be
v 1.0 1.0 1.0v 0.0 1.0 1.0v 0.0 0.0 1.0v 1.0 0.0 1.0v 1.0 1.0 0.0v 0.0 1.0 0.0v 0.0 0.0 0.0v 1.0 0.0 0.0
This results in the model shown below.
This is a unit cube 1m x 1m x 1m. The corners (vertices) are labled with their coordinates (1,0,1) and their vertex number (defined by their position in the list above {starting with vertex 1})
In order for the light in the model to interact with the cube correctly, each vertex needs to have a vertex normal, defined with the vn tag. While the Wavefront documentation states that the normal will be calculated from the points, this does not seem to always happen in the Gazebo Simulations.
vn 0.0 0.0 1.0vn 0.0 0.0 -1.0vn 1.0 0.0 0.0vn -1.0 0.0 0.0vn 0.0 1.0 0.0vn 0.0 -1.0 0.0
Since we are using all of a texture file (more on that later), we are not using any texture vertices.
The next section turns the vertices into a cube. The painted side is the side with the counter-clockwise rotation. You can think of this as a type of “right hand rule”, if your points are counter-clockwise, then normal is “up”. The vertex normal defines the reflective surface.
Starting with the top surface, we have to place the points counter clock wise, so the the painted face will show on the outside of the cube. This results in the order 1, 2, 3,4. Then the vertex normal will be facing in the z direction, which is vertex normal 1.
#topusemtl blue-texturef 1//1 2//1 3//1 4//1
For the bottom surface, the point go clock wise, so that the painted face is facing down. This results in the order 5, 8, 6, 7, with a vertex normal of 2.
#bottomusemtl blue-texturef 5//2 8//2 7//2 6//2
The side facing us as we come into the model is the negative x face, rotates in a counter clockwise way, and faces in negative x (vertex normal 6). The opposite side, rotates in a clockwise way, and faces vertex normal 5.
#x negative sideusemtl red-texturef 2//6 6//6 7//6 3//6#x positiveusemtl red-texturef 1//5 4//5 8//5 5//5
The remaining sides are the y facing sides#y positiveusemtl green-texturef 2//3 1//3 5//3 6//3#y negativeusemtl green-texturef 3//4 7//4 8//4 4//4 At this point, we have defined a cube, by defining the vertices. Then we define the normals that are used for the faces of the cubes, the vn terms, Then for each face of the cube, we tell the system which material to use to paint it, and how the image lines up with the face. So now we need to build an sdf that references the newly created object.
vn 0.0 0.0 1.0vn 0.0 0.0 -1.0vn 1.0 0.0 0.0vn -1.0 0.0 0.0vn 0.0 1.0 0.0vn 0.0 -1.0 0.0
Since we are using all of a texture file (more on that later), we are not using any texture vertices.
The next section turns the vertices into a cube. The painted side is the side with the counter-clockwise rotation. You can think of this as a type of “right hand rule”, if your points are counter-clockwise, then normal is “up”. The vertex normal defines the reflective surface.
Starting with the top surface, we have to place the points counter clock wise, so the the painted face will show on the outside of the cube. This results in the order 1, 2, 3,4. Then the vertex normal will be facing in the z direction, which is vertex normal 1.
#topusemtl blue-texturef 1//1 2//1 3//1 4//1
For the bottom surface, the point go clock wise, so that the painted face is facing down. This results in the order 5, 8, 6, 7, with a vertex normal of 2.
#bottomusemtl blue-texturef 5//2 8//2 7//2 6//2
The side facing us as we come into the model is the negative x face, rotates in a counter clockwise way, and faces in negative x (vertex normal 6). The opposite side, rotates in a clockwise way, and faces vertex normal 5.
#x negative sideusemtl red-texturef 2//6 6//6 7//6 3//6#x positiveusemtl red-texturef 1//5 4//5 8//5 5//5
The remaining sides are the y facing sides#y positiveusemtl green-texturef 2//3 1//3 5//3 6//3#y negativeusemtl green-texturef 3//4 7//4 8//4 4//4 At this point, we have defined a cube, by defining the vertices. Then we define the normals that are used for the faces of the cubes, the vn terms, Then for each face of the cube, we tell the system which material to use to paint it, and how the image lines up with the face. So now we need to build an sdf that references the newly created object.
Building the SDF
In the sdf, we have made a few changes to the original empty.sdf.
The completed file can be downloaded here : ManyColoredCube.sdfWe start with an empty.sdf file (If you don't know how to create one see this tutorial)
Since this just has a ground plane we have added the model of the many colored cube.
<model name='many_colored_cube'> <link name="cube"> <static>true</static> <pose> 0 0 0 0 0 0</pose> <visual name="cube_visual"> <geometry> <mesh> <scale>1 1 1</scale> <uri>manyColoredCube.obj</uri> </mesh> </geometry> </visual> </link> </model> Notice that all we really do is reference the Many Colored Cube obj file that we created above, and tell the simulator where to place it, and how to scale the model. The next this we do adjust the sun and add some additional lights.
<model name='many_colored_cube'> <link name="cube"> <static>true</static> <pose> 0 0 0 0 0 0</pose> <visual name="cube_visual"> <geometry> <mesh> <scale>1 1 1</scale> <uri>manyColoredCube.obj</uri> </mesh> </geometry> </visual> </link> </model> Notice that all we really do is reference the Many Colored Cube obj file that we created above, and tell the simulator where to place it, and how to scale the model. The next this we do adjust the sun and add some additional lights.
Adding some spotlights
<light name='luxo-left' type='spot'> <pose>0.0 6.0 3.0 -1.0 0.0 0.0</pose> <cast_shadows>true</cast_shadows> <intensity>1.0</intensity> <diffuse>0.800000012 0.800000012 0.800000012 1</diffuse> <specular>0.800000003 0.800000003 0.800000003 1</specular> <attenuation> <range>1000</range> <linear>0.01</linear> <constant>0.90000000000000002</constant> <quadratic>0.001</quadratic> </attenuation> <spot> <inner_angle>1</inner_angle> <outer_angle>1.5708</outer_angle> <falloff>0.5</falloff> </spot> </light>
This defines a spot light on the left side of the cube and 3m in the air (the 0.0 6.0 3.0 part of the pose) and it defines the rotation of the light to be a -1 radian rotation around the x-axis. Rotation is defined by roll, pitch, and yaw - roll is around the x-axis, pitch is around the y-axis, and yaw is around the z-axis.
We built three more spotlights,
one to the right side, <pose>0.0 -6.0 3.0 1.0 0.0 0.0</pose>
one in front <pose>-6.0 0.0 3.0 0.0 -1.0 0.0</pose> and
one behind the cube <pose>6.0 0.0 3.0 0.0 1.0 0.0</pose>.
Each one is pointing at the cube. This provides direct light on each face.
When you run the simulation, you should see:
You can see the four lights in the entity tree panel on the right.
The Entity tree lets you select and adjust the properties of the models in your simulation. For the lights, one property that you can change is whether they are on or off.
This lets you use the Gazebo interface to turn the lights out, to ensure that the simulation is lighting up correctly. All of the faces should have the correct shadows, and show the specular reflection of the appropriate light. Here, for example is the cube with only the front light on, showing the specular reflection and the shaded bottom surface.
You can change your viewpoint by using the mouse, the scroll wheel let's you zoom in and out, the normal motions move the viewpoint up/down or left/right, etc.
Here is the same setup, showing the shaded back of the cube, the specular reflection on the top of the cube, and the position of the lights.
So we turned all the lights except luxo-front off (including the sun) and moved the view point around to the back of the cube.
Several things to notice here. First the block is casting the correct shadows, second is the specular reflection on the top surface from the light source. The strength of the reflection is controlled by the Ks term in the materials file.
Next Steps
So far we have been using textures that are simply a solid color. Pretty boring, unless you are simulating BlocksWorld.In the next section we will take a more complex texture image and chop it up into parts to paint different surfaces.
What we did
This has been a long one, but we got a lot done:
We have:
1) Defined a new SDF that relies on an externally defined model of a cube.
2) Defined an object file that describes that cube, and its faces and materials
3) Defined a materials file that determines how those faces should look
4) Added texture files to be used by the materials file.
5) Added new lights to the scene to give the simulation shadows and highlights, just like in the real world.
Next step - Using complex textures
Previous Step | Current Step | Next Step |
None | Light and texture |