Creating a collection

Use the setup_collection service, of type warehouse/CreateCollection. If the collection does not exist, this will create a collection given the names of the db and collection, the ROS message package and type, and optionally, a set of fields to be indexed (the rest of the message is stored as a binary blob), each of which can either be auto-extracted from a nested field of the ROS message, or provided as additional metadata.

If the collection already exists in the db, it must have the same message type, md5sum, and indexed fields. This means that if the .msg file changes, the saved data in the warehouse will not be readable. We plan to eventually add a migration script for this case.

Pushing data

Let's say we've created a collection poses in db my_app, of type geometry_msgs/Pose, indexed by the x and y fields of the position. We can now push data in by publishing poses to warehouse/my_app/poses/insert. If the collection has metadata fields that are not automatically extractable from the message, these can be added by calling the add_metadata service.

Notifications

Each time a message is inserted, a notification is published, of type warehouse/InsertionNotification. In the above example, to warehouse/my_app/poses/notify. The notification includes the message sender id, the indexed fields (in yaml form), and the DB's unique id for this message.

Querying

We initiate a query by calling the initiate_query service, of type warehouse/InitiateQuery and specifying a db and collection name, and a structured query object, which is a bunch of conditions on the (indexed) message fields (right now, equality or arithmetic comparison predicates) or the additional fields _id and _creation_time. The warehouse then returns a query id and sets up a publication. In the example, if the returned query id was query_423, the topic would be warehouse/my_app/poses/query_423. Next, we pull messages by calling the pull_message service, of type warehouse/PullMessage, with the given query id. Each time we do this, a message is published on the query topic, until all the matching messages are exhausted. The order in which the messages are published can be customized using the order field of the service request.

The setup_collection service must have been called for the given collection before queries can be performed (i.e., if you kill the warehouse and bring it back up, you have to call setup_collection again).

Removing messages

You can remove all messages matching a given query using the remove_messages service.

Clearing an existing database

Use the drop_db service of type warehouse/DropDB. This will remove the database and all its collections, and unsubscribe to all corresponding topics.