Bitcom Flags: A Flexible Pattern for Complex Opreturn Protocols

As Bitcom based apps have become more sophisticated and feature rich, we have bumped up against the limits of what we can achieve with strictly positional push data arguments. What exactly do I mean by this? Let’s take a look at the B protocol for an example:

OP_RETURN
  19HxigV4QyBv3tHpQVcUEQyq1pzZVdoAut
  [Data]
  [Media Type]
  [Encoding (optional if no filename)]
  [Filename (optional)]

In this protocol, we have two required arguments which would be the data and media type. We also have two optional arguments, encoding and file name and at least one needs to be present. This is a very simple protocol and when a B server is parsing this data it is fairly trivial to determine whether the last argument is the file encoding or the file name (just search for a period).

However, what if we want more options? What if we would like to optionally set a title, timestamp, author, organization or other metadata? If we hardcode a specific ordering for this metadata, if we want to only set some fields, we would have to include a bunch of empty fields in our opreturn somehow.

JSON

The standard solution to this problem has been to use a JSON in a single push data and include whichever properties that you want. For example, here is some data from WeatherSV which is in a single push data:

"{\"t\":18.26,\"h\":48,\"p\":1022,\"c\":1,\"ws\":3.01,\"wd\":184}"

From my view there are several problems with this approach:

1. It’s wasteful. It may not seem like much but all of the quotation marks, brackets, colons and commas add up when you’re broadcasting billions of transactions.
2. Hard to query. since the data isn’t separated into different push datas it is very difficult to query for a specific field. You would need to perform a text search on the JSON which the current Planaria infrastructure is not set up to do well. And what if your JSON ends up being larger than 512 bytes? Then, it would be stored in bitfs and you wouldn’t be able to query it at all!
3. Limited data types. With JSON you can only store strings, bools and numbers. But what if you want to store an entire file as an argument? If you stored the binary as a string in JSON, your file would not be accessible in bitfs and parsing it would be a huge pain. What if you want to define your own protobuf instead of using strings?
4. Not very Unix-y. This is partly taste but it also has very practical implications. Unix philosophy is so powerful because it enables functionality that has yet to be imagined through simplicity and modularity. Bitcom has drawn much inspiration from this and to great effect.

Solution

To solve these problems we can take yet another page out of the Unix book with the concept of flags. Just like with a Unix cli, we can use flags to set options or arguments. Let’s look at the B protocol and add some functionality using flags:

OP_RETURN
  [B+ prefix]
  [Data]
  [Media Type]
  [Encoding (optional if no filename)]
  [Filename (optional)]
  -p
  [paymail address]
  -k
  [pubkey]
  -s
  [signature]

Using flags we can can set arguments in any order and we can include only the ones we want. And right out of the box, we can now query for flags using BOB:

{
  q:{
    find:{
      "out.tape.cell.s":"-p",
      "out.tape.cell.s":"jonathanaird@moneybutton.com"
    }
  }
}

There’s no limitation on how you can use flags. You could include all of the flags at the beginning, which is a common pattern from Unix:

OP_RETURN
  [B+ prefix] 
  -pks
  [Data]
  [Media Type]
  [Encoding (optional if no filename)]
  [Filename (optional)]
  [paymail address]
  [pubkey]
  [signature]

Or you could use long flags:

OP_RETURN
  [B+ prefix] 
  [Data]
  [Media Type]
  [Encoding (optional if no filename)]
  [Filename (optional)] 
  --paymail
  [paymail address]

If you want, you could combine the flag and the argument in one push data to make it easier to query:

OP_RETURN
  [B+ prefix] 
  [Data]
  [Media Type]
  [Encoding (optional if no filename)]
  [Filename (optional)] 
  --paymail=[paymail address]

The query for this flag would be:

{
  q:{
    find:{
      "out.tape.cell.s":"--paymail=jonathanaird@moneybutton.com",
    }
  }
}

Simple is good!

As you can see, this pattern is very simple, lightweight and completely unopinionated. You can use it in whichever way you want to accomplish what you need to with your app. As with many other things, this problem has been solved before and there’s no need to reinvent the wheel.