The user interface for converting files is coming along nicely. At first I was a little confused as to how to fit so many unrelated controls together in a manner that actually made sense. The answer seems to be tabs. The top half of the UI contains the controls that must be used in order to convert, while the bottom half contains a series of tabs, each controlling some optional facet of conversion. For example, one tab controls logging. Should a log be saved? Where? Should it be appended or overwrite the existing log. Another tab controls the axis translation. It allows the user to specify the way the axes were laid out during motion capture, and how they should be laid out in the output files. The advantage is that these optional, extra controls are all separated from the main conversion, but they are readily available if needed.
Linking the UI to the code did present a few problems. I wanted to keep the interface separate from the actual conversion, so that a user could stay with the command line version if they so chose. This required some re-factoring of the code, to turn the classes that handled conversion into a package that the command line or GUI version would pass information into to take care of the details. With a little extra effort I’ve accomplished it without any changes noticeable to the user.
The next order of business for me is to create the tool for creating subject-templates. The ones I’ve been using so far, I wrote by hand, but not everyone will want to do that as it is a tedious process and very easy to make a mistake (like leave out a marker). As opposed to making a totally different program for this, I’ve decided to add it into the GUI for conversion. In theory, this tool will only be used once in a while, as it is only paying attention to the labels of markers and if you use the same marker layout for every actor, the same template can be re-used on all of them. The vast majority of conversions will be done without ever looking at this tool (as long as any needed templates are created properly in advance). For these reasons I don’t want to include it on the main view, as it would take up space that could be used by more frequently accessed tools, but it should be easily accessible, as the files it creates will have important ramifications on conversions. In short, I’m going with tabs again, this time at the same level as the main converter pane. I debated putting it down with the others, but the creation of a template has no effect on a conversion, just the presence (or absence) of one. It is also going to be far more complicated than the options presented in those tabs and will likely require a sample view of what a file would look like with the template applied.
There are two things I noticed which are rather strange when rendering points.Shimmering borders
Points will appear to have a shimmering border when point smoothing is enabled (when points are rendered as circles rather than squares). The shimmering likely occurs because the points are moving. If the points are still, only small white artefacts are seen. Interestingly, when I zoom out using command-, the shimmering border is replaced with a nice clean white border. Why? I have no idea. The differences can clearly be seen in the following screen shots.
Frame rates
When I first open the mocap page, the frame rate hovers around 60 frames per second. The oddity is that when I zoom out, the frame rate drops. Again, I have no idea why. Should the frame rate drop at all? Continuing to zoom out, the frame rate begins to slowly increase. Here are some rough figures which I recorded:The numbers after zoom refer to how many times I zoomed in or out.
| Zoom | FPS |
|---|---|
| Zoom in 5 | 21 |
| Zoom in 4 | 27 |
| Zoom in 3 | 28 |
| Zoom in 2 | 31 |
| Zoom in 1 | 35 |
| Normal Size | 66 |
| Zoom out 1 | 24 |
| Zoom out 2 | 26 |
| Zoom out 3 | 30 |
| Zoom out 4 | 46 |
| Zoom out 5 | 66 |
Points and lines were designed to be grouped into lists because rendering a large number of points or lines could incur a performance hit. If there were disconnected points in a scene, every frame the renderer would have to iterate over all the points in the scene and copy their coordinates to a simple array. This array could then be passed to drawArrays(). By grouping the coordinates of points into a list within the scene class and exposing points as simple indices, the list would not have to be copied and can be directly passed into drawArrays().
I had not actually tested what the rendering hit would be, but realized it could be mimiced easily. For every frame, I took the contiguous list of coordinates and copied its elements into another newly allocated array. This roughly imitated what would be happening if the scene had to find all the points and copy each coordinate into an array before calling drawArrays(). I noticed there was a significant slowdown, from 60 FPS to 24 FPS, but it was not because of the code I just changed. Hitting refresh resized the canvas slightly, which I didn’t notice. This will be addressed in a blog shortly. In the end, the mocap demo does not seem to slow down, so to increase consistency within the library, PointList and LineList will be changed to Point and Line. Any performance issues which surface can be addressed in the library without the need to change any interfaces.
The next question is should each point be rendered individually? Would this slow down rendering? To test this, I just called drawArrays() for every point. The performance lagged a bit, there was a drop from about 66 FPS to 45 FPS. This means points may not need to be grouped into an array before being rendered. This could allow such flexibility as assigning each point its own attenuation and smoothing. However, this can also create some confusion.
The change will remove functions getPointLists(), getPoints(), getLineLists() and getLines() from the pickingObject class. This simplifies the object, but also makes it the user’s responsibility to check types since points, lines and models will all be in the list returned by getObjects(). This introduces the potential getType() method which each of our renderable object can implement.
// get the first object in the list.
var obj = pickingObject.getObjects()[0];
// get the type
if( obj.getType() == c3dl.const.POINT)
{
// ...
}
else if( obj.getType() == c3dl.const.LINE)
{
// ...
}
Next Steps
Convert c3dl.PointList into c3dl.Point and c3dl.LineList into c3dl.Line.As the title implies, I’ve spent that last little while juggling axes. I should probably clarify that that’s axes, the plural of axis, not axe.
I complained a little while ago about everyone having a different layout for their axes. Depending on who you talk to, x does not always point in the same direction, and one person will claim y points up, while another will say z does. It is easy enough to just swap the data around IF you know how the original data is laid out, and what layout you want in the end, but hard coding that would exclude anyone using a different system. So, I’ve added an option during conversion, where you tell the computer which axis pointed right, which one pointed up, and which one pointed out of the screen. Then you tell it which axis you want to point in each of those directions and it will dynamically sort out what data gets swapped. While this does exhibit a little bit of bias towards the right-hand-rule, (which would come out as +x+y+z), I can’t see any way of doing this without favouring one system or another. At least this way it’s asking what axis (+ or – and x,y, or z) points in arbitrary fixed directions. Almost everyone can agree that up is up, you just have to tell the converter what axis your system would point in that direction and likewise for right and out.
This also allows you play with the data a bit, turning a simple stroll across the floor holding a length of rope:
into a mighty struggle to climb up the side of a building.
Now that the converter has this option too, the command line is getting a little awkward, so I’m going to start working on a nice user interface to help make it easier to perform complex conversions.
When picking was added to the library, we did not anticipate anything else being picked other than 3d meshes. To obtain the data calculated by the internal picking function, users have to write a callback function:/**
@param {int} buttonUsed Defines what mouse button was pressed when the user
clicked on the canvas.
@param ObjectsHit {Array} Array of integers which are the array indices of
the objects in the object list array of the scene.
@param Scene {c3dl.Scene} Scene which was clicked. There may be many
canvases on the page rendering their own scene.
*/
onPick( buttonUsed, ObjectsHit, Scene)
{
//...
}
The user would then check to see if any objects were hit and possibly which button was used. They could then perform any necessary logic.This worked, but only up until now. Points need the same picking functionality, however points were not designed as objects, but as part of a point list. This difference creates a problem in the picking callback interface (How can they be returned?). Additionally, if a new type of object is added to the library, the potential for it to be pickable must be considered, therefore an extensible interface is required.
One way to address this problem is to simply keep adding arguments to the picking callback, but this approach is not very elegant. Another possibility is to make the individual points and lines classes themselves, however this would require more changes to the PointList and LineList classes which would need re-testing. Not changing the picking callback function now would also force any future object to be implemented in a certain fashion.
I’m proposing a change to the picking callback function. This would break old code, but would create an extensible interface to handle future types of objects and functionality. Instead of having the callback receive three arguments, it would accept a single pickingObject.
onPick(pickingObj)
{
// get an array of references to objects which have been picked.
var objectsPicked = pickingObj.getObjects();
// we can query the picking object and find out which button was used
// for the pick.
if( pickingObj.getButtonUsed() == 1)
{
//...
}
// we can get the number of objects/models the user picked by
// getting the length of the array of references.
if( objectsPicked.length > 0)
{
// ...
}
// how many pointlists had at least one of their points picked?
var pointLists = pickingObj.getPointLists();
// iterate over all the point lists which have been picked.
for( var i in pointLists )
{
// since we have the reference, we can call the pointlists
// methods easily without needing to get the reference
// from the scene.
// ie. scene.getObj(i)
if(pointList[i].getName() == "whatever")
{
// for this particular pointlist, we'll need to
// get which points were picked. getPoints()
// will return the indices of all the picked points
// for pointList at i.
var pickedPointsIndices = pickingObj.getPoints(i);
// let's say we want to change all the picked
//points to red.
for(var j in pickedPointsIndices)
{
pointList[i].setPointColor(
pickedPointsIndices[j], [0,1,0,1]);
}
}
}
}
If this implemented, some of the functions of the pickingObject will include:
// get an array of references of pointLists which had at least
//one of their points picked.
getPointLists();
// get an array of indices of points which were picked for pointList at 'i'.
getPoints(int i)
// get the mouse button which was used for the pick.
getButtonUsed();
// get an array of references of objects picked
getObjects();
// get the canvas which was picked. From the canvas we can get the scene.
getCanvas();
// get an array of references of lineLists which had at least
// one of their lines picked.
getLineLists();
// get an array of indices of lines which were picked from LineList at
// index i of the list of linelists
getLines(i);
