Here you'll find an assorted mix of content from yours truly. I post about a lot
of things, but primarily
Day 27: Rubin's Bitcoin Advent Calendar
24 Dec 2021
Welcome to day 27 of my Bitcoin Advent Calendar. You can see an index of all
the posts here or subscribe at
judica.org/join to get new posts in your inbox
I know, I know.
God forbid, a roadmap. People hate roadmaps. As I’ve noted
before:
Bitcoin Eschews Roadmaps and Agendas.
I provide this maxim to make clear that this document is by no means an official
roadmap, narrative, or prioritization. However, it is my own assessment of what
the current most pragmatic approach to upgrading Bitcoin is, based on my
understanding of the state of outstanding proposals and their interactions.
My priorities in producing this are to open a discussion on potential new
features, risk minimization, and pragmatic design for Bitcoin.
If you didn’t click, I definitely recommend reading the quoted
post in conjunction with this one. As well as, if
you’re a first time visitor to the Advent Calendar, the preceding 26 posts.
In contrast to my prior post, this roadmap is going to be less about full
justifications and unbiased weightings and tallyings of sentiments and more just
me spitting out a timeline we could introduce changes on. It’s not a final
answer, and in no way authoritative, but it’s a launch point for a discussion
that has to happen in some way at some point in order to advance a soft-fork.
Consider this as being posted for the sake of public review. If you disagree
with this, let me know why! But please no attacks just for the act of discussing
the topic of soft-fork activation.
So buckle up here’s how we could make Bitcoin kick ass in 2022 and beyond:
2022: The Year of the Covenant
BIP-119 Timeline
CheckTemplateVerify is getting close to ready to go. There are numerous
supporters (listed on utxos.org), few detractors,
and a bumper crop of amazing use cases (did you read the calendar?) waiting for
us on the other side of CTV activation. The major critiques are that we might want
something that does ‘more’ than CTV itself, or to include it in a bundle of
things.
My take: we’re not Gordon Gecko. Greed is Not Good. CTV represents a clean, well
contained, unproblematic upgrade that’s going to deliver hella functionality in
service of scaling, decentralization, self custody, and privacy. Let’s secure
the bag for Bitcoin users everywhere and make it happen. We can always do more,
later, informed by what extensions we need for rapidly maturing tools like
Sapio. CTV is also technically specified and implemented sufficiently – a view
I’ve confirmed with a couple other devs – that it is able to be considered for
release.
What would have to happen to release CTV?
- More signalers would need to be on
utxos.org/signals or other platforms to demonstrate
interest and demand for CTV, ideally explaining which use cases are important to
them and why. Every voice counts for consensus. There is no list long enough to
capture what it would mean to have consensus, so there is not some threshold
that implies a ‘go ahead’, but N+1 is clearly better than N.
- More “regular contributors” would need to spend time reviewing the code and BIP to
assure themselves of correctness and safety. Nothing can move forward with out,
no matter the count of casual contributors. Many regular contributors don’t want
to ‘get political’ and look at forks. Fortunately, while all consensus changes
are complex, CTV is a very tiny and easy to review change in comparison with
SegWit or Taproot (more similar to CheckLockTimeVerify – a couple hundred lines
of consensus code, a couple hundred lines of non consenus code, and a couple
thousand lines of tests, no cryptographic primitives). NOTE: This is a big
if! Every contributor has the right to review, and ACK or provide a reasoned
NACK. Even if everyone else is excited about something doesn’t mean there isn’t
space for new thought-through dissent. At the end of the article, I discuss
some concrete next steps to ensure more developer review occurs.
- We would need to merge the implementation. This is simple, but enough ACKs have to
come in and rebases on other subsequent changes to get it in. This can
happen ahead of ‘full consensus’ since there are no deployment parameters,
but aids in increasing the testing priority of CTV.
- We would need to get a majority of miners/pools primed to accept the upgrade.
- Major alternative implementation maintainers (e.g., BTCD, Knots) should show
willingness to implement or accept patches for the new rules (although it’s a
soft-fork, this is good to do).
- We would need to decide on release parameters for the implementation.
- We would need to merge and release a client with release parameters
- The client would needs to lock-in by a supermajority of miners signalling.
- Then, and only then, would CTV be fully available.
What’s the Maximally Aggressive/Optimistic Timeline?
- Soft Signals / Developer Review: 2-3 months required to get ACKs on the
implementation, assuming no major changes required.
- Merge: Instant, just whenever there are ACKs against the implementation being
safe and matching BIP. Usually, enough ACKs for a PR is 2 regular contributors,
the comitter, and a maintainer, but for consensus changes there is no threshold
at which Bitcoin considers a change sufficiently peer reviewed. A consensus
change should see higher quality reviews, as well as external consensus that the
change is desired.
- Getting miners primed to signal: ~20% of pools are on
utxos.org/signals, more should be coming on board
soon. Don’t expect this to take additional time if in conjunction with Developer
review.
- Debating Timelines: ~1 month to agree on a release timeline
- Preparing a release: ~1 month to do release candidates and testing.
- Speedy Trial: 3 months of signalling, 3 months of waiting for lock-in to active.
Overall, it could look like this:
On March 15th developers reach agreement on merging BIP-119’s implementation.
On April 15th, agreement is reached on release parameters for signalling from ~
June 1st to ~September 1st. The activation height would be November 10th. A client
is prepared, and tested, and released. No issues are found. The miners signal at
some point in the 3 month window above the threshold. CTV locks-in. Developers
can prep wallet software for deeper integration. CTV activates before
Thanksgiving, avoiding the “dire straits” of
thanksgiving-hanukkah-christmas-chinese-new-year-valentines-day season.
but that’s basically identical to Taproot’s timeline?
Exactly. If we act on this timeline starting in early January 2022, it is
possible to meet an almost identical timeline for CTV as Taproot.
Part of why it works is that the next major release is scheduled for 2022-04-01.
Soft forks are usually released as a minor patch on top of a few recent major
releases. So CTV could be:
- Code included in 23.0 for CTV, no activation parameters
- Activation parameters in 23.1, and backported to 22.x, 21.x, and (really this is up to maintainers how far to backport!).
- 23.1 released before June 1st, 2022.
What could go wrong?
- Concern: There could be a small tweak to CTV that makes it marginally
better, and it’s worth adding some extra review time as a result.
Rebuttal:
CTV is highly design specific, so it’s unlikely there could be a change needed,
but not impossible. Changes would be unlikely to be large, though, e.g. perhaps
comitting to the same fields in a different order. Taproot saw some small
changes about a month before being merged.
- Concern: Release process has a delay due to an issue uncovered in non-CTV code.
Rebuttal:
Soft fork releases are usually a minor patch onto an existing version, so it’s
unlikely that there would be a new bug, but the release would still be planned
and could resume as soon as patched. Speedy Trial’s ‘delayed activation’ also
helps with providing more time for (non-consensus) bug fixes in the client
between lock-in and activation.
- Concern: Release process has a delay due to an issue uncovered in CTV’s code.
Rebuttal:
If the issue is a bug, it would merit more strict scrutiny on the code and
tests (which are pretty comprehensive currently) as to how they could be passing
with an issue like that. Once patched and reviewed, it’s still possible to merge
and release. However, the issue could also be a conceptual bug (e.g., here’s how
to do recursive covenants with CTV), which would certainly delay (and I’d be the
first to say it!) continuing with CTV at the present time. The former issue is
likely more likely than the latter, and that risk is defrayed by thorough code
review and testing.
- Concern: There’s not enough developer consensus conceptually.
Rebuttal:
There are many developers supporting on
utxos.org/signals. I’ve also made an effort to
reach out to a variety of developers who are not on the site to seek out
dissent, and I do not think there are any current safety concerns, which is
positive for developers who might not Ack but don’t have a ‘this is a grave
mistake’ Nack up their sleeves. Again, there’s no well defined threshold for
what is “enough”.
- Concern: Miners don’t want CTV and don’t signal for it.
Rebuttal:
_20% of the hash pools already say they do want it, and more should be joining
that list soon™. Should that not happen, there could be a UASF effort to ensure
the activation of BIP-119 if users want CTV strongly enough. _
- Concern: Disagreement on Activation mechanism.
Rebuttal:
Taproot’s Speedy Trial (ST) worked very well. Big Success. No need to mess with
the recipe. UASF-clienters know how to make the mods to the activation logic for
whatever they want as a competing client. But perhaps that’s not a comfortable
status quo, so this is certainly one to watch. I’ve made my own remarks on what
order of operations I think ST/UASF/etc should go in
here,
but there’s not consensus on this topic. Notably, there is some dissent from
reusing ST at all. While important, activation logic is a secondary (but still
critical) concern to the decision to accept the current specification and
implementation of CTV in the first place, and discussion on that can proceed
in parallel to progress on consensus on the implementation.
And Then What?
After the release of CTV Soft Fork Client, what goes next?
- Jeremy takes a vacation for a month.
- Sapio continues to improve dramatically, as detailed in the last post.
- Work begins on new Opcodes.
- More fun applications get built on CTV/Sapio (for examples, review the entire series of
the Advent Calendar, but a few of my favorites are Payment
Pools,
Vaults, and Bonded
Oracles).
- Non-Interactive Lightning Channel BOLT drops.
- Payment Pool spec drops with LN integration.
- (Privacy + Scale)++
New Opcodes you say?
Yep, new opcodes. As soon as CTV is merged, there are some new features that
could be tooled into BIPs without much controversy, given their simplicity:
- New Math Opcodes
- OP_AMOUNT
- OP_CSFS
These are “universally agreed on” bits of functionality that have very little
wiggle room in how they might be specified. Therefore their implementation,
testing, and release is mostly mechanical. This makes them good bets for a
concrete timeline because they’re a rote-development task with few research
dependencies and easily agreed on design.
With hard work, these could be reviewed and tested in time for Speedy Trial
signalling in June 2023 (+1 year), with realllllly hard work 6 months earlier
(but there are conflicting priorities for developer’s time – e.g., building
stuff with/for CTV and Taproot – that make that super unlikely).
What about Anyprevout
Anyprevout is 1000000% not ruled out here for advancing to the consensus stage
in 2022. There are a couple things happening on Anyprevout that make me suspect
it might be more primed towards early-mid 2024 best case.
- Taproot upgraded channels taking some steam away from Eltoo with AJ’s
proposal to do state compaction via PTLCs.
- Disagreement over no justice transactions in the community for high value
channels.
- Open research on specific Eltoo protocol implementation / need for other
tools (like SIGHASH_BUNDLE).
- Lack of an open PR and test suite for it.
- CSFS + CTV permitting a version of Eltoo.
All of these reasons point to it being highly unlikely APO could be finished on
the timeline presented for CTV, but also that given a desire to see a working LN
client (ensuring the protocol is sound end-to-end without modifications) leads
to additional engineering complexity.
I’ve heard on the rumour vine some excellent engineers might start working on an
Eltoo implementation, I believe that work to be of tremendous value to gain
consensus on deploying anyprevout.
Given that a couple years seems like the best case for a set of upgrades around
APO to deliver Eltoo, if we have the ability to deliver CTV now, it is a good
asymmetric bet for delivering utility to Bitcoin’s users.
Suppose I’m wrong, and Anyprevout really could get done in 2022. Shouldn’t it
be ‘next’ and CTV can wait?
The notion that any soft-fork is “next” for consideration and blocks any other
from being considered is somewhat contrary to the support for parallel
soft-forks with distinct version bits used for signalling. The possibility of
deploying soft forks on independent bits means things can be ultimately
“proposed” whenever they’re mature enough to enter the “final stage” and the
timing fits with Bitcoin’s general release schedule. The chief counterarguments
to this are twofold: One, review resources are finite and we can’t even think
about \(>1\) thing at a time; Two, it’s bad if the protocol is simultaneously
deploying \(N\) upgrades and any of them might fail, leading to \(2^N\)
potential protocol states. It’s actually the other way around: if things can
proceed in parallel we can get more review, because developers won’t feel that
reviewing others work has the potential to slow down their own, and we’ll be
more certain that the upgrades we release are strongly desired by the community
(i.e., will end in a UASF if not activated due to miner apathy).
What about TapLeafUpdateVerify
If I had to estimate, I’d say TLUV needs another year or so of people ‘stewing’
on the possibilities and design before it could move forward. It would then
probably need at least a year of tinkering to get a well-accepted
implementation, and then a year for release process. So let’s say best case it
could be a 2024 thing, more likely 2025 given Anyprevout is seen as higher
priority for the engineering work.
What about Inherited IDs (IIDs)?
They didn’t make the cut in the earlier piece since they’re a bit more abstract,
but I’d posit that IIDs could also approach a 2024 timeline were the developer
of the idea to spend a heckin’ big amount of time on advocacy for the concept in
2022, and present a clean implementation of the concept (& demonstration of
overheads) by 2023. This is complicated by the issue that he would have to also
solve issues with requiring a new index, such as helping assist assumeutxo
compatibility, write reindexing logic from genesis (or somehow exclude old
coins?), and also work on utreexo compatibility. Those all seem tractable,
but very hard to do for someone who is not a full-time long-time contributor
to the project, but I’m a believer that the job could be done given the quality
of the insights in the IIDs paper.
Parallel Developments?
In parallel to the above, there are some things that I think are very cool
that should remain under development, but don’t have a clear path to inclusion
presently.
The first is Transaction Sponsorship, which could substantially
simplify many protocols on top of Bitcoin, especially things like Eltoo and CTV
stuff. We can’t predict that too well because it will depend on what developers
end up running up against with current best practices around CPFP/RBF, but
I suspect it might become popular if a small group of developers prioritizes
such an approach as a unified front. Since it’s technically simple, it wouldn’t
take much time to implement, but because there’s very little consensus for
it right now it’s not fit for inclusion in a roadmap.
The next is Simplicity. Simplicity could completely change how Bitcoin
scripting works and is super duper exciting. I guess it’s going out on Elements
sometime soon? However, it’s stupidly complicated for most devs to fully
understand (although it is called simplicity…), so it would take a fair
amount of time (read: years) before the implementation could be sufficiently
independently reviewed by the intersection of qualified Bitcoin and Programming
Language theory reviewers. The interesting thing about Simplicity with respect
to this roadmap is that because it’s so general, as soon as it seems like
Simplicity would be on Bitcoin within ~1.5 years, it’s likely all other
scripting soft fork developers would “stop” development and focus on
deployment as Simplicity Jets.
Concrete Next Steps
Should this plan not seem feasible to the near unanimity of the community for us
the deliver on (e.g., with +2-3 months of the suggested timeline), I don’t think
there is another good opportunity to enact on an activation of CTV and the best
course of action would be to delay +1 year. There could be a UASF for it, since
there is strong user demand for CTV, but I wouldn’t personally lead the charge
on that beyond ensuring that the BIP-119 code remains up-to-date and is
implemented correctly.
Assuming someone doesn’t make a bold argument that gives me or the aggregate
current supporters of CTV pause, I plan to begin holding a fortnightly CTV
review session to iron out any details, additional testing, code review, and
activation plans. I would also like to help a third party host a review
club, although the focus of Review Club is more
educational for the reviewers as opposed to for the purpose of formally
reviewing the code for merge.
You’ve now reached the end of Rubin’s Bitcoin Advent Calendar, 2021! Congrats.
wc -w $(ls | grep advent) tells me that means you read 50,773 words total of
me rambling on about covenants. That’s PhD thesis length! I really hope you
found something to enjoy.
A special thanks to all who helped review this post in particular and provided
feedback in ensuring I’ve done an OK job capturing the complexity of gaining
consensus. To the extent that I haven’t succeeded in representing your
perspectives here, I take sole responsibility for the inadequacy. Your
constructive feedback on how to improve is welcomed. A further special thanks to
Ryan Gentry who encouraged me to produce the series in the first place, and to
Sarah Satoshi for her support as I battled my way through this task.
As for me, I’m off to sip some Eggnog and eat some holiday cookies! I’ll take a
little bit of a break, and be back in the New Year following up on next steps
and dropping fresh content for y’all as always.
Merry Christmas and Happy New Years,
Jeremy
Day 26: Rubin's Bitcoin Advent Calendar
23 Dec 2021
Welcome to day 26 of my Bitcoin Advent Calendar. You can see an index of all
the posts here or subscribe at
judica.org/join to get new posts in your inbox
Sapio began as little more than a slide in presentations I would give on what
BIP-119 could be with the support of a programming environment.
While my conceptions of what could be built with CTV were about on-par with
where they are today, the tools available were very clunky. You can see
one of the original diagrams from the C++ Code below (code here).
/*
* [ vault_to_vault n ]
* |a |b
* (0) (1)
* /\ /\
* wait + maturity / \ / \ wait + step_period (attached)
* [ hot_to_hot ] / [vault_to_cold] [vault_to_vault n - 1]
* / | |
* [hot_to_cold] (0) (1)
* . .
* . .
* .
* wait + step_period * n (attached)
* [vault_to_vault 0]
* |
* (0)
* */
When I presented it on February 1st, 2020 at the CTV workshop, one of the last
Bitcoin events before the pandemic began, I knew I had to present a more
concrete vision on how smart contracts could be built, rather than spit-and-glue
C++.
And so I included slide inspired by the hand rolled smart contracts I was making
in C++ about how the technique could be generalized.
a slide at the feb. 2020 CTV workshop.
This is a research project I am interested in pursuing. We have a few
scripting languages. We have Dan here who wrote Ivy. There’s also Balzac and
Solidity. It’s poorly named, but it actually has some really interesting ideas
that aren’t in other scripting languages for bitcoin. There’s some notion of
participants with private state they are storing, and has some notion of
interactivity. I also threw up there BlueSpec which is like a VHDL competitor
that is pretty neat. If you think about it, there’s a software layer for
controlling vaults but there’s also a hardware layer- so how can we describe
transactional hardware? You wind up with similar controls as a BlueSpec-like
language where you are creating plumbing and a description of how things are
moving rather than something that has imperative state.
As an example, one of the properties with OP_CHECKTEMPLATEVERIFY that is cool is
the narrow window of a standard OP_CHECKTEMPLATEVERIFY script is that you have
this composability property where for a given model that goes to arbitrary
address outputs, you can just throw another one inside it. As the outputs of
that annuinity, you can put undo send as the outputs of that annunity, and you
can tag on more and more programs into that if you want. On this slide is a
templately description of a language, which helps if someone else given the same
program description can verify that the hash is the same for the given inputs.
Instead of sending you a million branches, you can generate them yourselves.
Q: Is there code for your metascript thing?
A: No. It’s just an idea I’ve been toying with. I got relatively close to
metascript just writing in C++ just writing with C++ templates. Then I got ready
to kill myself doing that because writing templates.
I’d discussed similar slides at earlier events, usually included it as an
example of something someone else could build while I kept pluggin’ away
working on Bitcoin Core. Eventually I realized that me claiming that CTV could
be used to do these things and CTV demonstrably doing these things was an
obstacle… I needed to make the prototype.
So I set off hacking together a prototype… I spent an afternoon or two mucking
around in C++ only to realize that wasn’t going to happen, and then I switched
to python. Python was a great choice, and after quite a bit of time pluggin
away, I had something that actually worked.
There was a moment of magic where I connected it to the Sapio Studio (which
actually existed long before Sapio as just a visualizer for a set of
transactions) to load a contract written in python and… it worked. Really
well. It felt incredible. I began talking more about what Sapio could do,
thinking through some of the problems and solutions. I even gave a talk in the
metaverse with Udi where I dropped my virtual laser pointer on the ground for a
minute and couldn’t pick it back up.
there were more people sitting further back I swear…
At a certain point I decided to start working on cleaning up Sapio by using
Python’s gradual typing and trying to convert to using Miniscript instead of my
custom-made Sapio script fragment builder. I was neck deep in type refactors and
then I had a deep realization:
Python FUCKING SUCKS
I was incredibly greatful for the rapid iterating that python allowed me, but I
realized at this point that it was going to be nearly impossible to make Sapio
actually good and not just a toy working in python. What had started as a proof
of concept for someone to do “for real” had gotten good enough conceptually but
was bursting at the seams and was not going to ever yield something production grade.
So I did a reset.
I started building Sapio in Rust, re-implementing (but much more cleanly) the
paradigms that I had previously developed, relying on well tested and type
rust-bitcoin libraries instead of janky test-framework python code from Bitcoin
Core.
And one day – almost like Deja Vu – I plugged the Sapio Rust served in place
of the Python one for the Sapio Studio and everything Just Worked™.
Since then, Sapio has only matured and gotten better and better with each
passing month. I’ve added tons of new features, improved the underlying
architecture, made the ABIs more well defined, built stronger intergrations with
the GUI, and more. Today, Sapio is actually usable on mainnet (if you’re an
expert at least), and I’ve used to do congestion control payments and art
projects.
Sapio has helped me cut through the content for this Advent Calendar like a
swiss army knife. It’s actually becoming a pretty decent tool!
What’s Next
Sapio needs to get to the next level. There are many major areas of work in the
pipeline. You might think of this as “Sapio is so incomplete it’s not ready”.
I think of it more as Sapio is just getting started:
Upgrade to Taproot
Support for Taproot in rust-bitcoin and rust-miniscript is coming! Once that
lands, I need to rebase (and maybe upstream?) some Sapio features, and then
update the compiler to always use Taproot outputs!
Improve Sapio CTV Emulators with Taproot (if ctv seems to be slow)
Once taproot lands, the multi-sig federated oracle designs can be set up to do a
MuSig Schnorr signature instead of just a bare multi-sig, allowing wider federations.
This engineering work only really matters if CTV seems unlikely, but could
be useful to have anyways for future purposes & extensions to Sapio.
Build out some “full featured” applications backed by Sapio
Right now Sapio works for making little Applets, but there is no “end to end”
software (e.g., vaults) made in Sapio.
This requires work on both the Sapio Studio front and potentially on a website
for things like Sapio NFTs.
This will push the boundaries on integrating Sapio into real applications.
One particular feature that would be great is ‘auto-signing’ on valid state
transitions proposed at a certain continuation point. For example, if we have an
NFT we want to dutch auction, we should be able to have a sapio module running
that is happy to spit out the relevant transactions with a nice API.
More advanced programming of transactions
Currently the Sapio transaction builder from within Sapio branches is a bit
limited in what it can express.
Augmenting it with the ability to more fully describe partially signed bitcoin
transactions (PSBTs), e.g., when used in the continuation context would
advance the state of the art.
Key Management
Relatedly, it would be really useful if you could tell Sapio how to access a
signing module and have sapio use that external key to generate the appropriate
signatures for the state transitions.
Right now every time a module is used the entire thing has to reload which is
super slow.
It should be possible to have a single server instances that manages a cache of
all modules, greatly boosting performance, especially for recursive cross-module
calling contracts.
Client Side Verification Caching Across Module Boundaries
For things like NFTs that we saw, we have to always re-compile the entire
history of the NFT to do state transitions. However, many of these state
transitions are just doing the exact same thing every time, re-verifying from
genesis. We should be able to cache the compilations at well defined boundaries.
Currently Sapio has the ability to define various interfaces for composing Sapio contracts cleanly.
But we don’t have a great way of proving what we want a contract to do v.s. what it does.
This work will start as being able to express more properties about a module in
it’s “manifest”, and will culminate in being able to check the validity of compositions
of contracts based on them.
If anyones looking to do a PhD thesis on Sapio, this is prolly it.
More Module Types
Right now all WASM modules are for generating a ‘Compiled’ contract from a JSON.
I would love to add defined module types to cover all the standard components like:
- Miniscript Fragments
- Then Functions
- Continuation Functions
- Transaction Template Generators
- Trait Verifiers
and more.
More Emulator Types
Sapio works today with CTV then functions because the functionality can be
emulated by a federation of signers.
Figuring out how to cleanly extend the signing server paradigm to a number of
different types of Covenant proposal (e.g. APO, TLUV, CAT) or other opcodes
could add a lot of value for making Sapio the defacto platform and proving
ground for upgrades to Bitcoin.
Unifying Then and Continuation
Then and Continuation really are the same thing conceptually, it would be
Standard Library of useful applications / Plugins
Title says it all. There should be more off the shelf things to use!
Wallet Stack for Sapio Contracts
The Sapio Studio is rapidly imporving as a system for engaging with Sapio
contracts, but it’s lacking in the management and storage of all of a user’s
contracts. We can, and will, do a better job with this.
This includes being able to make watchtower-like functionality for Sapio. It’s
coming, but will likely require a lot of new features to the language spec to
express if this then that monitoring conditions. And with features comes
complexity.
You can see hints to this in the slide earlier, being able to react to e.g.
confirmation or mempool events.
This also includes dramatic improvements needed to the GUI so that users
can understand much more deeply what contracts are doing.
Binary Releases
Right now Sapio is just DIY to build and run it yourself, but we really ought to
work towards stable binary releases to quickstart users.
Visual Programming for Building Sapio Contracts
When composing together Sapio modules, I would really like for it to be kind
of labview like interface for adding data or other modules, where you can
“typecheck” that the wires (module names) are composing into slots they work.
If that sounded like gibberish, I just mean that I want to be able to plug in a
module hash for a NFT Sale contract into my NFT contract and have the UX give
me real time feedback if I should be able to do that.
Sapio Package Manager / Deterministic Builds / Signers
As more modules become commonly used (e.g., like NFTs) we want to ensure all
users can build an exact copy themselves (Deterministic Builds) or that they
can find a build signed by a number of respected code signers (Signers). While
we can use Cargo / crates.io for the package management of our actual rust code,
we need something to distribute the sapio wasm blobs nicely!
Testing
Writing tests for Sapio contracts is hard as heck. We need some bold ideas on
what sorts of properties would be useful to test for. Fortunately, a lot of
properties we care about (like not wasting money) can be checked by the sapio
library to be safe, but we still want to be able to ensure we never lose a
users funds out of Sapio.
#### Getting CTV Locked in
Duh!
If I get even a tenth of this completed 2022 will be a good year for Sapio.
But I plan to get it all done.
Perhaps with your support – if you’re an engineer or funder interested
in helping propel this, please reach out!
p.s. safe holiday travels!
Day 25: Rubin's Bitcoin Advent Calendar
22 Dec 2021
Welcome to day 25 of my Bitcoin Advent Calendar. You can see an index of all
the posts here or subscribe at
judica.org/join to get new posts in your inbox
The title of this article is a joke. Gotcha!
Decentralized Autonomous Organization is pretty much what’s called an orphan
initialism.
So while DAO doesn’t really mean anything is decentralized, autonomous, or an
organization, but the term DAO has stuck around anyways. Even moreso than NFT!
More or less, DAOs are just fancy multisigs. But they’ve been used for all sorts
of things, ranging from attempting to buy the US Constitution as a group,
investing in startups, buying Ross Ulbricht’s NFTs, or maybe even buying my
undies.
This post has some required reading. You have to have read through at least up
to payment pools in the advent calendar, but
ideally you’d have read all the posts…
So how will fancy-multisigs save Bitcoin? In this post we’ll work through an
example of building a DAO to fund Bitcoin Core Developers, like a Bitcoin native
Gitcoin competitor.
What do we want from our DAO:
The DAO will serve three functions:
- To Add members who pay in
- To make Payments if a Majority of the payers by value vote
- To mint commemorative NFTs
Majority rules
DAOs are little democracies, and as such we need a voting scheme to do rule
changes whereby a threshold (e.g., 51%) decides what happens next. We have two
options, we can either count individuals as equal, or we can weight by amount of
funds contributed. We can do any threshold we like, it’s just “this many people
could steal the whole pot”.
For this post, we’ll do the weighted by funds contributed because that feels
closer to what’s happening in Ethereum land. Unfortunately a couple components
around generated arbitrary weighted signatures just “aren’t quite there” or have
messy tradeoffs so we won’t consider those – yet. Instead we’ll just make a silly
limit: we will allow at most 24 participants.
Implementing a DAO
First let’s define the basics. A DAO should have Members who each are ID’d by a
key and have an amount of votes.
#[derive(Deserialize, JsonSchema, Serialize, Clone)]
struct Member {
relative_votes: u64,
}
#[derive(Deserialize, JsonSchema, Clone)]
struct Dao {
/// # Pool Members
/// map of all initial balances as PK to BTC
members: BTreeMap<PublicKey, Member>,
/// The current sequence number (for authenticating state updates)
sequence: u64,
}
impl Contract for Dao {
declare! {updatable<Proposal>, Self::hold_vote}
}
Members can hold a vote on a proposal of some kind. Let’s do proposals
that can make payments, mint NFTs, or add some noobs:
/// New Update message for generating a transaction from.
#[derive(Deserialize, JsonSchema, Serialize)]
enum Proposal {
/// # Payments
/// A mapping of public key in members to signed list of payouts with a fee rate.
Payments {
payments: BTreeMap<PublicKey, AmountU64>,
// Some purpose for this proposal, as a String.
reason: String,
},
/// # Mint
/// Make some NFTs
Mint {
minting_module: SapioHostAPI<Mint_NFT_Trait_Version_0_1_0>,
mint_data: Mint_NFT_Trait_Version_0_1_0,
},
/// # Add People
Add {
noobs: BTreeMap<PublicKey, Member>,
},
None,
}
/// required...
impl Default for Proposal {
fn default() -> Self {
Proposal::None
}
}
impl StatefulArgumentsTrait for Proposal {}
/// helper for rust type system issue
fn default_coerce(k: <Dao as Contract>::StatefulArguments) -> Result<Proposal, CompilationError> {
Ok(k)
}
Now we can implement the main logic of the DAO. We want it to compute keys for
the majority to rule, and we want it to allow a majority to vote on a Proposal.
Note how when we make a payment, unlike in the Payment Pool, we decrease all
member’s proportional ownership in the pool, so that new owners are not
disadvantaged. But we could change that, to time-weight how long members have
been part of the DAO as well, or give people ‘special voting weight’
disconnected from money added. It’s really up to whatever you want…
We’ll implement the logic for each type of proposal (minting, adding, or paying).
impl Dao {
/// Sum Up all the balances
fn total(&self) -> Amount {
Amount::from_sat(self.members.iter().map(|e| e.1.relative_votes).sum::<u64>())
}
/// all signed the transaction!
#[guard]
fn majority_rules(self, _ctx: Context) {
let ppl = self
.members
.iter()
.map(|(m, d)| (m.clone(), d.relative_votes))
.collect();
// TODO: we should probably make guards return Result...
key_groups_to_clause(
&compute_key_groups(self.total().as_sat() / 2, ppl).expect("Well Formed"),
)
}
/// This Function will create a proposed transaction that is safe to sign
/// given a list of data from participants.
#[continuation(
web_api,
guarded_by = "[Self::majority_rules]",
coerce_args = "default_coerce"
)]
fn hold_vote(self, ctx: Context, update: Proposal) {
// don't allow empty updates.
match update {
Proposal::None => empty(),
Proposal::Mint {
minting_module,
mint_data,
} => {
let key = minting_module.key;
// let's now compile a new 'mint' of the NFT
let new_nft_contract = Ok(CreateArgs {
context: ContextualArguments {
amount: ctx.funds(),
network: ctx.network,
effects: Default::default(),
},
arguments: mint_impl::Versions::Mint_NFT_Trait_Version_0_1_0(mint_data),
})
.and_then(serde_json::to_value)
.map(|args| create_contract_by_key(&key, args, Amount::from_sat(0)))
.map_err(|_| CompilationError::TerminateCompilation)?
.ok_or(CompilationError::TerminateCompilation)?;
let f = ctx.funds();
ctx.template()
.add_output(f, self, None)?
.add_output(Amount::from_sat(0), &new_nft_contract, None)?
.into()
}
Proposal::Add { mut noobs } => {
let adding = Amount::from_sat(noobs.values().map(|m| m.relative_votes).sum());
let mut new = self.clone();
noobs.iter_mut().for_each(|(pk, m)| {
new.members
.entry(*pk)
.and_modify(|e| e.relative_votes += m.relative_votes)
.or_insert(m.clone());
});
let f = ctx.funds();
ctx.template()
.add_sequence()
.add_amount(adding)
.add_output(f, self, None)?
.add_output(Amount::from_sat(0), &new, None)?
.into()
}
Proposal::Payments { payments } => {
if payments.is_empty() {
return empty();
}
// collect members with updated balances here
let spent = payments
.values()
.cloned()
.map(Amount::from)
.fold(Amount::from_sat(0), |a, b| a + b.into());
let balance = 1.0 - (spent.as_btc() / self.total().as_btc());
let mut new_members = self.members.clone();
new_members.values_mut().for_each(|m| {
m.relative_votes = (m.relative_votes as f64 * balance).round() as u64;
});
// for each payment...
// Send any leftover funds to a new pool
let change = Dao {
members: new_members,
sequence: self.sequence + 1,
};
let mut tmpl = ctx.template().add_output(change.total(), &change, None)?;
// optional: we could commit to the reason somewhere in metadata
// e.g. a tapleaf branch... we don't do this here because meh.
for (key, amount) in payments {
tmpl = tmpl.add_output(amount.try_into()?, &key, None)?;
}
tmpl.into()
}
}
}
}
REGISTER![Dao, "logo.png"];
Lastly, we need some super special sneaky algorithm fun to implement signing authorities
based on majority value. As noted, special uses of FROST could replace this, or future
research on better weighted key protocols.
For now, we limit ourselves to 25 keys so that compilation isn’t too slow. We can afford
having hundreds of thousands or millions of groups because of Taproot :).
fn key_groups_to_clause<T>(v: &(Vec<(PublicKey, T)>, Vec<u32>)) -> Clause {
Clause::Threshold(
1,
v.1.iter()
.map(|m| {
Clause::And(
v.0.iter()
.enumerate()
.filter_map(|(i, (k, _))| {
if m & (1 << i) != 0 {
Some(k.clone())
} else {
None
}
})
.map(Clause::Key)
.collect(),
)
})
.collect(),
)
}
fn compute_key_groups(
threshold: u64,
mut el: Vec<(PublicKey, u64)>,
) -> Result<(Vec<(PublicKey, u64)>, Vec<u32>), CompilationError> {
if el.len() > 25 || el.is_empty() {
return Err(CompilationError::TerminateCompilation);
}
// sort for stable ordering
el.sort();
// The bitmasks for which keys to participate
let mut sets: Vec<u32> = vec![];
// BEGIN ALGORITHM:
// if we see a bit set out of range, we can stop.
let fail_if_set = ((!0) >> el.len()) << el.len();
// we know that 0 elements is invalid, we need up to el.len()
for i in 1u32..=el.len() as u32 {
// get the first member of our permutation
let mut ct = element_0(i);
// if any bits are set in the failure zone stop
while ct & fail_if_set == 0 {
// compute the sum of the elements in this mask
let sum: u64 = (0..el.len())
.map(|i| if ct & (1 << i) != 0 { el[i].1 } else { 0 })
.sum::<u64>();
// this set is a candidate!
if sum >= threshold {
// subtract the smallest value (this is why we sorted) -- if it
// fails it is not a minimal set because there exists a passing
// set without this element.
// note: trailing zeros is guaranteed to be in bounds
if sum - el[ct.trailing_zeros() as usize].1 < threshold {
// it did fail, so save it
sets.push(ct);
}
}
// get the next ct
ct = next_perm(ct);
}
}
Ok((el, sets))
}
/// Adapted from https://www.alexbowe.com/popcount-permutations/
///
/// Compute the lexicographically next bit permutation
/// Taken from http://graphics.stanford.edu/~seander/bithacks.html
fn next_perm(v: u32) -> u32 {
let t: u32 = v | (v - 1); // t gets v's least significant 0 bits set to 1
// Next set to 1 the most significant bit to change,
// set to 0 the least significant ones, and add the necessary 1 bits.
let w: u32 = (t + 1) | (((!t & (!t).wrapping_neg()) - 1) >> (v.trailing_zeros() + 1));
w
}
/// Generates first permutation with a given amount of set bits, which is
/// used to generate the rest.
fn element_0(c: u32) -> u32 {
return (1 << c) - 1;
}
All done! Not too bad huh? I think you’re really getting the hang of this thing!
Using a DAO
Now that we have this DAO we can get together a group of people and share a UTXO.
With that shared balance, we can get everyone in some kind of chat room and ‘govern’
what proposals folks want to vote on.
In particular, I would be very excited to see DAOs emerge for funding Bitcoin
Developers. This type of structure can potentially help folks communally
allocate capital. Often times the biggest barrier is finding deals that make
sense, and DAOs would enable you to share with a group of friends and they could
make decisions for you.
It would even be possible to create DAOs on behalf of third parties and fund
them. For example, let’s say I get PKs for 10 devs I like and put a 10 BTC into
it and set the shares up so that there is a ‘leader’ with 30%, and the rest
split 70% of voting shares. The leader could just steal the money with another
21%, but would they? I hope not! Instead, they can vote on good things as
intended. It’d also be possible for the DAO creator to embed an ‘oversight
comittee’ that can yank the funds if not being used.
Minting NFTs is kind of a cool feature since anyone can see they came from the
DAO if they track the DAO’s state updates (conceivably these get published for
auditing). NFTs could be issued as medals of honor for devs who follow their
grants. Or, if you really like NFTs, they could be used to issue software
licenses in exchange for contributing funds to the DAO operators.
Does this need CTV?
Nope. Just a fancy multisig, right?
Where CTV is useful is if we want to vote on proposals to put things into CTV
contracts, like subscriptions to developer grants, opening channels, etc.
Imagine the developer gets a contract where they get paid out every week, but
there is a auditing comittee that can be used to terminate the subscription and
return funds to the DAO if misbehavior is detected.
While you don’t need CTV in the DAO backbone, it would help open up new use
cases.
It would also be possible to add some ‘liveness’ smooth degradations of the DAO,
whereby half the majority (e.g., if majority is 50%, 25%) could vote that the
DAO is dead, and after a period of time for the majority recovery, distribute
the funds on a pre-comitted schedule.
We don’t show that here, but it wouldn’t be too hard now would it?
Generalizing
One could go ahead and implement a DAO trait that all DAOs could share and build
a common UX for managing DAOs with a wide variety of custom logic…
It’d also be possible to have a DAO backbone which is a single UTXO, and have
other UTXOs ‘owned’ by the DAO that can get merged in later as a proposal. This
way contributions to the DAO don’t always require a state update from the DAO
itself.
For future work :)
Day 24: Rubin's Bitcoin Advent Calendar
21 Dec 2021
Welcome to day 24 of my Bitcoin Advent Calendar. You can see an index of all
the posts here or subscribe at
judica.org/join to get new posts in your inbox
Today’s post is near and dear to my heart – years ago I put up an interest form
for powswap.com, but as I went down the rabbit hole I
realized how badly I wanted generic tooling to automate the building of these
which is partly what led to Sapio!
So therefore it’s very exciting to show you the basics of powswap in Sapio. You
can see how bad the early version was
here. If
you want to contrast life with Sapio and without.
What is a Powswap?
The basic idea of Powswap is super simple. It is a contract that measures a
block surplus or deficit – a Block Delta Contract (BDC). A BDC allows
counterparties to bet on statements like “at the end of 6 months, we will be +/-
1000 blocks against the current expected number of blocks”, and program a payoff
curve based on the binary outcome of that. The block delta should be – and this
is a matter for the analysts to price on and model – correlated with changes in
hashrate.
WHO THE F CARES
Well imagine you are about to buy a new fancy mining rig to mine with. But you
have a moment of doubt – what if everyone else is doing that right now too?
You could buy hashrate derivatives where you win money if the hashrate increases
and lose if it stays the same.
This would de-risk your investment in mining.
You can also lever-up and increase profit if you’re adding a lot of hashrate,
doubling down that hashrate goes up, but let’s not entertain the degens shall
we.
OK OK How can I do it?
One could imagine making a BDC based on the Oracle system we saw in yesterday’s
post. But the magic of Powswap is that we will do this without using any oracle
whatsoever, just measuring the blocks directly.
How do we do this?
The answer is actually really simple. Suppose Alice wants to get 1 Bitcoin if
100 blocks are missing at the end of the week (the 28th, let’s say expected 1000
blocks), and Bob wants to win 1 Bitcoin if they are actually there.
All we have to do is have Alice and Bob agree to a multisig to deposit 0.5 BTC
each to, and then pre-sign from it two transactions:
- If the date is the 28th at noon and the height is greater than 1000, Bob gets paid 1 BTC
- If the date the date is the 28th at noon + 8 hours, Alice gets paid 1 BTC
Let’s think it through:
Suppose that Alice is right and blocks are 100 short by
noon.
In the next 8 hours, only 48 blocks should be mined (and probably less, if the
hashrate has actually decreased).
After that point, Alice has 8 more hours (again, probably more if hashrate
actually decreased) to broadcast and claim her BTC.
Suppose that Bob is right and blocks reach 1000 at noon. Bob has 8 hours to
claim the BTC before Alice can.
Where this is a bit wonky is that the result is metastable. Let’s assume that
neither Alice nor Bob is right: The deficit is 50 blocks short.
At noon, Bob cannot claim. But in 8 hours he can! But also in 8 hours Alice can
claim too.
So who wins?
The answer is either! Using a POWSWAP you either want to be really right or
really wrong.
We’ll see some cool results around why this might not be a huge deal later.
Let’s flip the powswap around now, for a surplus of blocks. Bob thinks the
blocks will be 1000, Alice thinks 1100.
- If the date is the 28th at noon + 8 hours, Bob gets paid 1 BTC
- If the date the date is the 28th at noon and there are 1100 blocks, Alice gets paid 1 BTC
Under this model if Alice is right there should be that many blocks by that
time, and if Bob is right there should not be and a resaonable amount of time
later Bob can claim.
It’s a bit harder to see, but we can even implement this logic more simply as just:
- If we reach +1 week, give Bob 1 BTC
- If we reach +1100 blocks, give Alice 1 BTC
Then, if a week goes by first without seeing 1100 blocks, Bob can claim. If the
1100 blocks show well before the week is up, then Alice can claim. If neither
are really right then it’s metastable and either could win.
We are not going there!
There are a myriad of different combinations of locktimes and heights that you
can use to do this correctly, we won’t focus too much on that in this post, and
we’ll let our contract users decide what parameters they want. Let the analysts
figure out what the right combo of locktimes and stuff is to hedge different
risks. They should get paid for something, right?
One of the wrinkles is that the less time you have in your contract, the more
metastable it is. The more time you have, expecially across difficulty
adjustments, the more the deficits can be erased.
Implementing a Powswap
Is CTV Required?
In the example I gave above, it is not! However, if you have CTV then one party
can unilaterally open a hashrate derivative for other parties, and that matters
quite a lot!
This means that when we do implement it, we will use then because if you
want the pre-signature version you can use CTV Emulators.
First we’ll start by writing some code to be able to describe the locktimes
under which some outcome is considered “resolved”. We’ll write a container type
(the data we actually need) and then we’ll write a verifier type that makes
for a convenient API for human input. It’s kind of gross, so you can skip
the verifier type code and just imagine you put in the correct parameters.
/// `ContractVariant` ensures that we either set a Relative Height and Absolute
/// Time or a Relative Time and Absolute Height, the two valid combinations, or
/// just one.
///
/// Note these are unlocking conditions for each participant.
///
/// Validity is ensured through smart constructor
#[derive(JsonSchema, Deserialize, Clone, Copy)]
#[serde(try_from = "ValidContractVariant")]
pub struct ContractVariant(Option<AnyRelTimeLock>, Option<AnyAbsTimeLock>);
/// In order to test for coherence here, we should convert
/// ValidContractVariant to ContractVariant.
///
/// The coherence rules should match one ruleset of:
/// - a single type of TimeLock (Relative Height, Relative Time, Absolute Time,
/// Absolute Height)
/// - a mixed TimeLock of just Relative Height/Absolute Time or just Relative
/// Time/Absolute Height
#[derive(JsonSchema, Deserialize, Clone)]
struct ValidContractVariant(Vec<AnyTimeLock>);
impl TryFrom<ValidContractVariant> for ContractVariant {
type Error = CompilationError;
fn try_from(vcv: ValidContractVariant) -> Result<Self, Self::Error> {
let abs: Vec<_> = vcv
.0
.iter()
.filter_map(|v| {
if let AnyTimeLock::A(a) = v {
Some(a)
} else {
None
}
})
.collect();
let rel: Vec<_> = vcv
.0
.iter()
.filter_map(|v| {
if let AnyTimeLock::R(r) = v {
Some(r)
} else {
None
}
})
.collect();
let all_rh = rel.iter().all(|v| matches!(v, AnyRelTimeLock::RH(c)));
let all_rt = rel.iter().all(|v| matches!(v, AnyRelTimeLock::RT(c)));
#[derive(Debug)]
struct LocalError(&'static str);
impl std::fmt::Display for LocalError {
fn fmt(
&self,
f: &mut std::fmt::Formatter<'_>,
) -> std::result::Result<(), std::fmt::Error> {
self.0.fmt(f)
}
}
impl std::error::Error for LocalError {}
if !(all_rh || all_rt) {
return Err(CompilationError::custom(LocalError(
"Must have some timelock set!",
)));
}
let all_ah = abs.iter().all(|v| matches!(v, AnyAbsTimeLock::AH(c)));
let all_at = abs.iter().all(|v| matches!(v, AnyAbsTimeLock::AT(c)));
if !(all_ah || all_at) {
return Err(CompilationError::custom(LocalError(
"Incoherent Absolute Timelocks (mixed height/time)",
)));
}
let relative = rel.iter().max_by_key(|v| AnyRelTimeLock::get(v)).cloned();
let absolute = abs.iter().max_by_key(|v| AnyAbsTimeLock::get(v)).cloned();
if matches!((relative, absolute), (None, None)) {
return Err(CompilationError::custom(LocalError(
"Must have some timelock set!",
)));
}
if (all_rt && all_at) || (all_rh && all_rt) {
return Err(CompilationError::custom(LocalError(
"Must mix {Relative,Absolute} Height and Absolute time!",
)));
}
Ok(ContractVariant(relative.cloned(), absolute.cloned()))
}
}
impl ContractVariant {
fn get_relative(&self) -> AnyRelTimeLock {
self.0.unwrap_or(RelTime::from(0).into())
}
fn get_abs(&self) -> AnyAbsTimeLock {
self.1.unwrap_or(AbsHeight::try_from(0).unwrap().into())
}
}
With that out of the way, let’s now define our contract data:
/// Instructions for a Payment from an outcome
#[derive(JsonSchema, Deserialize, Clone)]
pub struct Pays {
sats: AmountU64,
to: PublicKey,
}
/// A `Outcome` is a contract where
#[derive(JsonSchema, Deserialize, Clone)]
pub struct Outcome {
/// # Variant
/// if the base is time or height for the relative leg.
unlocks_if: ContractVariant,
/// # Outcome
/// Payments to make (should be >= 1)
outcome: Vec<Pays>,
}
/// A `PowSwap` is a contract where
#[derive(JsonSchema, Deserialize, Clone)]
pub struct PowSwap {
/// # Parties
pub outcomes: [Outcome; 2],
/// # Cooperate Key
coop: Vec<PublicKey>,
}
impl Contract for PowSwap {
declare! {then, Self::payoff}
declare! {finish, Self::cooperate}
}
As you can see, it’s pretty simple. We just need a set of keys to ‘opt out’ of
the on-chain execution, and a set of outcomes and their unlocking conditions.
We can pay an arbitrary number of parties.
Now to finish, let’s implement the logic. It’s really simple, we just create the
(2) transactions and assign the sequences/locktimes properly.
impl PowSwap {
#[guard]
fn cooperate(self, ctx: Context) {
Clause::And(self.coop.iter().cloned().map(Clause::Key).collect())
}
fn make_payoffs(&self, ctx: Context, payments: &[Pays]) -> Result<Builder, CompilationError> {
let mut bld = ctx.template();
for Pays { sats, to } in payments {
bld = bld.add_output(sats.clone().into(), to, None)?;
}
Ok(bld)
}
#[then]
fn payoff(self, mut base_ctx: Context) {
let mut ret: Vec<Result<Template, _>> = vec![];
for (i, path) in self.outcomes.iter().enumerate() {
let ctx = base_ctx.derive_num(i as u64)?;
let v = self
.make_payoffs(ctx, &path.outcome)?
.set_sequence(-1, path.unlocks_if.get_relative())?
.set_lock_time(path.unlocks_if.get_abs())?
.into();
ret.push(Ok(v));
}
Ok(Box::new(ret.into_iter()))
}
}
That wasn’t so bad now, was it?
Using PowSwap
We already said we’re not going to analyze the profit of these contracts, but I
want to give a couple cool ways to use these.
When to Cooperate?
One thing that I think would be important to settling a hashrate derivative would be
to set it for, say, 6 months forecast and then try to roll the strategy at 3 months cooperatively.
This way you don’t have trouble with metastability as you and your counterparty
can update forecasts and re-enter the contract, or go separate ways.
TAKE IT TO THE LIMIT
Well what if instead of settling on-chain, you nested these in LN channels? And
then every microsecond you don’t see a block being advertised and broadcast, you
update your probabilities and try to adjust with your counterparty. It becomes
pretty neat becuase you essentially make a hashrate perpetual where if your
counterparty dies then you settle on-chain (if they’re really dead, you just
win), but you can update your forecasts on whatever frequency you want. All
trustlessly.
This opens the door for HFT-ing information about the rate of block production.
Knowing a block is mined and getting it relayed to you before your counterparty
gives you an edge in trading.
Maybe this pays for really really good block relaying infrastructure?
GALAXY BRAIN ME
Hey, it’s me. Your old friend Decentralized Coordination Free Mining Pools. What
if we made – using CTV – the channels/payouts by default resolve into some sort
of hashrate future, and we had an automated hedging market maker that could incorporate
your desired side of a trade from old hash shares into opening new positions for you every block.
If it was in channels you could immediately turn these into hashrate perps.
If you’re a miner and you mine, say, 2 blocks a day, then you can usually expect
to be able to settle your own metastable hashrate derivatives as long as the
metastable window isn’t smaller than ~12 hours. This means that while normie
pleb users might struggle with closing their derivatives, miner-to-miner
hashrate derivatives should be actually pretty safe if you stay in your
bounds.
Can we really do this? How effective is it? Honestly I have no idea I just think
it’s mind-blowing.
Gimme all My Options
This idea composes beautifully with the options we saw yesterday. What if I want
the option for the next week to open up a 6 month hashrate contract with you?
Just toss it into an Expiring Under Funded Option contract and you got it. And
because we represented these as Dutch Auctionable NFTs, you can advertise the
position you’re willing to open to the network and take the best offer for this
option.
Sapio composes. Legit forreal.
DeFi is coming to Bitcoin.
And it’s going to help with securing the base layer of Bitcoin by permitting
trustless financialization of investments in hashrate.
Have a great day. P.s. now is a good time to join
utxos.org/signals if you think CTV is a great next
step in Bitcoin Development’s journey.
Day 23: Rubin's Bitcoin Advent Calendar
20 Dec 2021
Welcome to day 23 of my Bitcoin Advent Calendar. You can see an index of all
the posts here or subscribe at
judica.org/join to get new posts in your inbox
In today’s post we’re going to talk about derivatives and options. Hoooo Boy!
Let’s define an Option:
An option is a contract that gives the holder the right to take an action to the
detriment of a counterparty. Options can be created for payment.
For example, I might say to you, “hey! I heard you’re pretty good at coming up
with memes. I’d like to pay you $10 for the option to buy the next meme you make
supporting OP_CTV for $100.” You might say “OK”, and then make a meme.
I think it’s awful and I tell you to go away. I’m out the $10, but not $110! I
can’t post the meme though or my friends will think I’m lame for sharing Right
Clicked Content. Or, if I think it’s great, I can pay you the $100 and then
I’m off to the races.
There are a few different types of Option contract to think about:
Call v.s. Put.
A Call option we get the right to buy something later for a fixed price (like a preorder).
A Put option we get the right to sell something later for a fixed price (like insurance).
Here’s how to remember it:
Think about a nice puppy. You Call the puppy to you, and give him a cookie, head
pat, and a “good boy”.
Think about a naughty puppy. You take away your chewed up sneaker, and Put her
in her crate. “Bad girl”.
American v. European
American options you can settle at any time before it “expires”.
European options you must settle during a specific window after it expires, and
before the window expires.
Think of European options like a restaurant reservation. You can no-show if you want,
but you can show up between 7:00 and 7:15 and be seated.
American options are like a hold on jacket you think is beautiful at Saks Fifth
Ave. You could come and buy it later today, tomorrow even! But wait a week and
someone else will buy it because they put it back on the rack.
Collateralized v. Non Collateralized
A collateralized option means that the asset is actually there.
For example, think of Jerry Seinfeld renting a car. He reserved the car (buying
a call option), but they didn’t have a car when he showed up. They knew how to
take the reservation, but not keep it.
This is an example of a ‘naked short sale’ of the car rental, because it wasn’t
backed by an actual car to rent.
I don’t have a proof, but we can’t really build that kind of thing in Bitcoin.
However, imagine if Jery had to, in order to make his reservation, make a
deposit for the entire value of the reservation. Sure, he could get a refund,
but then he would have an opportunity cost of which options. Imagine you’re
taking your partner out for a suprise dinner that’s going to cost $1000, but you
don’t know which one is going to be better so you reserve two restaurants and
cancel one you decide not to go to for a 1% penalty. If you had to deposit, it would
cost you $10 for the no-show, but you’d have to put up $2000 to hold the reservations!
If instead, you paid both restaurants $10 up front, then you would only need to lock
up $20 instead. Much more efficient!
These, we will build.
The Optimal Strategy for Pricing Options
Just kidding. I have no idea. It’s a complex subject, but some people are OK at
it. This post is just plumbing.
What’s Derivative?
Your humor…. burnnnn. Just kidding.
A derivative is uhhh… well. It’s anything that isn’t the thing?
A derivative is a way of taking a real thing (e.g. a ton of corn, an NFT, an
Apple Stock) and then either wrapping it or observing it in some other financial
product.
In fact, Options themselves are Derivatives! Wacky, right? It’s a thing (e.g. a
car) and then the right to buy that car has a price and a value that is a
function of what the car is, but a lot of other factors too. So the option isn’t
the car, but it’s connected.
An option is a “Real Derivative” because it is actually connected to the car
that is bought or sold. But we can also make “Synthetic Derivatives” that just
measure some external quantity (somehow) and then give you some amount of value
in return. For example, I could make a synthetic option that mints an NFT of a
car instead of the actual car. Or I could make a synthetic derivative that
measures the price of the car over the last month and gives me that value in
Bitcoin at the end of the month.
For synthetics, they have to be over collateralized to cover all outcomes. E.g.,
if we expect the car to be $50,000, but it might go to $100,000, we have to lock
up $100,000. And if the price is $200,000, well our max profit is $100k then.
The Options we saw earlier were very binary in outcome. Synthetic derivatives
like these can emulate any function, discontinuous or continuous. So you could
have a contract, for example, that pays out on a sinusoidal wave based on the
car price. Trippy.
Wheres the info come from?
Well, multiple places. We could get it from a third party (maybe using an
attestation chain of some sort?), or there are certain ways it could be
self-referential (like for powswap).
Let’s See Some Code
Synthetic Derivatives
I love the children equally, so let’s start with Derivatives now.
First let’s define an Oracle Interface who provides us data. All the Oracle does
is, given a Symbol (some request), gives us a Clause they will help us satisfy
if the Symbol is true, and something else if it is false. Imagine a symbol that
you can query an oracle for questions such as “is a Bitcoin worth more than
$50k”.
/// Placeholder type for a standard way of looking up a stock symbol; can be defined more
/// concretely but should have a human readable string representation.
pub type Symbol = String;
/// Oracle is a generic wrapper for any logic to get a pair of binary clauses.
/// It can be based on hash preimage, federated signers, or key revealing.
/// The Trait Object can be responsible for network requests/caching.
pub trait Oracle {
/// returns keys (price lo, price hi) for the given query
fn get_key_lt_gte(&self, t: &Symbol, price: i64) -> (Clause, Clause);
}
Now let’s define a threshold oracle – we wouldn’t want to trust just one
lousy oracle, so let’s trust M out of N of them!
/// An Oracle can also be "composed" into a threshold scheme with other
/// oracles quite easily as below...
///
/// Under *certain* circumstances, composition could be optimized (e.g., schnorr keys)
pub struct ThresholdOracle {
/// the list of price oracles to consult
pub oracles: Vec<Box<dyn Oracle>>,
/// how many oracles must agree
pub thresh: usize,
}
impl Oracle for ThresholdOracle {
fn get_key_lt_gte(&self, t: &Symbol, price: i64) -> (Clause, Clause) {
let (l, r) = self
.oracles
.iter()
.map(|o| o.get_key_lt_gte(t, price))
.unzip();
(
Clause::Threshold(self.thresh, l),
Clause::Threshold(self.thresh, r),
)
}
}
The underlying clauses could really be anything… we can even (with some tweaks
to Sapio I’d LOVE to get working, but need to engineer) make this represent
Discrete Log Oracles with 2 counterparties and an external Oracle. If that means
something to you, good, otherwise you can ignore that remark.
Now, let’s define a Generic framework for any outcome. The key insight we need
to have is that we can ask the oracle a bunch of greater-than-or-less-than
questions and build up a binary tree of transactions to settle at the right price.
To start, let’s define some basic stuff for a ‘GenericBet’.
/// A GenericBet takes a sorted list of outcomes and a cached table of
/// oracle lookups and assembles a binary contract tree for the GenericBet
pub struct GenericBet {
amount: Amount,
outcomes: Vec<(i64, Template)>,
oracle: Rc<HashMap<i64, (Clause, Clause)>>,
cooperate: Clause,
}
impl Contract for GenericBet {
declare!(then, Self::pay_gte, Self::pay_lt, Self::oracle_no_show);
}
But where do we get the list of price to outcome and price to oracle clause from?
We need an external data source, right?
We’ll define some arguments and then a way of turning those arguments into a precise GenericBet.
We do it this way so that we can have GenericBetArguments accept
a non-deterministic oracle server type, and then GenericBet itself could live in WASM and be fully deterministic.
/// To setup a GenericBet select an amount, a list of outcomes, and an oracle.
/// The outcomes do not need to be sorted but must be unique.
pub struct GenericBetArguments<'a> {
amount: Amount,
outcomes: Vec<(i64, Template)>,
oracle: &'a dyn Oracle,
cooperate: Clause,
symbol: Symbol,
}
/// We can then convert the arguments into a specific contract instance
impl<'a> From<GenericBetArguments<'a>> for GenericBet {
fn from(mut v: GenericBetArguments<'a>) -> GenericBet {
// Make sure the outcomes are sorted for the binary tree
v.outcomes.sort_by_key(|(i, _)| *i);
// Cache locally all calls to the oracle
let mut h = HashMap::new();
for (k, _) in v.outcomes.iter() {
let r = v.oracle.get_key_lt_gte(&v.symbol, *k);
h.insert(*k, r);
}
GenericBet {
amount: v.amount,
outcomes: v.outcomes,
oracle: Rc::new(h),
cooperate: v.cooperate,
}
}
}
Now, we’ll implement the logic behind a generic bet:
Basically, we do a binary search over all the outcomes to find the middle, and
if the price is greater, we send to that one. Otherwise, the other one.
By winnowing through all of these outcomes recrusively, we are able
to resolve a single price:action pair and settle the contract.
impl GenericBet {
/// The oracle price kyes for this part of the tree is in the middle of the range.
fn price(&self, b: bool) -> Clause {
let v = &self.oracle[&self.outcomes[self.outcomes.len() / 2].0];
if b {
v.1.clone()
} else {
v.0.clone()
}
}
fn recurse_over(
&self,
range: std::ops::Range<usize>,
ctx: sapio::contract::Context,
) -> Result<Option<Template>, CompilationError> {
match &self.outcomes[range] {
[] => return Ok(None),
[(_, a)] => Ok(Some(a.clone())),
sl => Ok(Some(
ctx.template()
.add_output(
self.amount.into(),
&GenericBet {
amount: self.amount,
outcomes: sl.into(),
oracle: self.oracle.clone(),
cooperate: self.cooperate.clone(),
},
None,
)?
.into(),
)),
}
}
/// Action when the price is greater than or equal to the price in the middle
#[guard]
fn gte(self, _ctx: Context) {
self.price(true)
}
#[then(guarded_by = "[Self::gte]")]
fn pay_gte(self, ctx: sapio::Context) {
if let Some(tmpl) = self.recurse_over(self.outcomes.len() / 2..self.outcomes.len(), ctx)? {
Ok(Box::new(std::iter::once(Ok(tmpl))))
} else {
Ok(Box::new(std::iter::empty()))
}
}
/// Action when the price is less than or equal to the price in the middle
#[guard]
fn lt(self, _ctx: Context) {
self.price(false)
}
#[then(guarded_by = "[Self::lt]")]
fn pay_lt(self, ctx: sapio::Context) {
if let Some(tmpl) = self.recurse_over(0..self.outcomes.len() / 2, ctx)? {
Ok(Box::new(std::iter::once(Ok(tmpl))))
} else {
Ok(Box::new(std::iter::empty()))
}
}
/// Allow for both parties to cooperative close
#[guard]
fn cooperate(self, _ctx: Context) {
self.cooperate.clone()
}
#[then]
fn oracle_no_show(self, _ctx: Context) {
// elided for simplicity: unilateral close initiation after certain
// relative delay if oracle doesn't reveal data
}
}
This is, by itself, useless. But now that we have it we can implement now any
payoff curve we want to. I’ll just show one example and leave it as “homework”
for you to build others. We’ll start with the humble risk-reversal, which can be
used to stablize a Bitcoin against the dollar. You can think of it as a Bitcoin
“low pass” filter: you’ll still see big price swings, but not little ones. See below:
Value of BTC in Asset
|
| /
| a /
| <------ b /
| -------------> /
| ----------------------
| / ^
| / |
| / current price
| /
--------------------------------------------------- price of BTC in Asset
Amount of BTC
|
|-------
| \
| \ ^
| \ \
| \ \
| \ \
| \ \ a
| \ \
| \ \
| \ \
| \ \
| \ <- current price
| \ \
| \ \
| \ \
| \ \ b
| \ \
| \ \
| \ \
| \ \
| \ \
| \ \
| \ \
| \ v
| \
| --------------
|
--------------------------------------------------- price of BTC in Asset
In this case, Operator would be providing enough Bitcoin (Y) for a user’s funds (X) such that:
\((current - a)*(X+Y) = current * X\)
or
\(Y * current = a * (X + Y)\)
and would be seeing a potential bitcoin gain (Z) of
\((current + b) * (X - Z) = current * X\)
or
\(Z = b * X / (b + current)\)
or \(Z (current + b)\) dollars.
Operator can profit on the contract by:
- selecting carefully parameters a and b
- charging a premium
- charging a fee (& rehypothecating the position)
Similar to our GenericBetArguments we’ll compile this to a GenericBet to hide all the Network-y stuff.
First, let us define a couple APIs we need for the maker and taker of a contract (e.g., the person offering dollar stabilization and the person needing it).
/// An API for the Operator Service:
pub trait OperatorApi {
/// Return Operator's Oracle
fn get_oracle(&self) -> &dyn Oracle;
/// Get a fresh key clause for Operator signing (could be a multisig etc)
fn get_key(&self) -> Clause;
/// Get a contract for a receivable amount. Allows Operator to direct funds to e.g.
/// cold storage contracts
fn receive_payment(&self, amount: Amount) -> Compiled;
}
/// An API for the Counterparty
pub trait UserApi {
/// Get a fresh key clause for user signing (could be a multisig etc)
fn get_key(&self) -> Clause;
/// Get a contract for a receivable amount. Allows Userto direct funds to e.g.
/// cold storage contracts
fn receive_payment(&self, amount: Amount) -> Compiled;
}
Now, let us define the Arguments to a Risk Reversal:
//! RiskReversal represents a specific contract where we specify a set of price ranges that we
//! want to keep purchasing power flat within.
pub struct RiskReversal<'a> {
amount: Amount,
/// the current price in dollars with one_unit precision
current_price_x_one_unit: u64,
/// price multipliers rationals (lo, hi) and (a,b) = a/b
/// e.g. ((7, 91), (1, 10)) computes from price - price*7/91 to price + price*1/10
range: ((u64, u64), (u64, u64)),
// ignore the
operator_api: &'a dyn apis::OperatorApi,
user_api: &'a dyn apis::UserApi,
symbol: Symbol,
ctx: Context,
}
Lastly, a bunch of complicated logic to turn those arguments
into a price curve for a GenericBetArguments that then gets turned
into a GenericBet:
const ONE_UNIT: u64 = 10_000;
impl<'a> TryFrom<RiskReversal<'a>> for GenericBetArguments<'a> {
type Error = CompilationError;
fn try_from(mut v: RiskReversal<'a>) -> Result<Self, Self::Error> {
let key = v.operator_api.get_key();
let user = v.user_api.get_key();
let mut outcomes = vec![];
let current_price = v.current_price_x_one_unit;
// TODO: Can Customize this logic to for arbitrary curves or grids
// bottom and top are floor/ceil for where our contract operates
let bottom =
((current_price - (current_price * v.range.0 .0) / v.range.0 .1) / ONE_UNIT) * ONE_UNIT;
let top = (((current_price + (current_price * v.range.1 .0) / v.range.1 .1) + ONE_UNIT
- 1)
/ ONE_UNIT)
* ONE_UNIT;
// The max amount of BTC the contract needs to meet obligations
let max_amount_bitcoin = (v.amount * current_price) / bottom;
// represents an overflow
if bottom > current_price || top < current_price {
return Err(CompilationError::TerminateCompilation);
}
let mut strike_ctx = v.ctx.derive_str(Arc::new("strike".into()))?;
// Increment 1 dollar per step
for strike in (bottom..=top).step_by(ONE_UNIT as usize) {
// Value Conservation Property:
// strike * (amount + delta) == amount * current price
// strike * (pay to user) == amount * current price
// pay to user == amount * current price / strike
let profit = (v.amount * current_price) / strike;
let refund = max_amount_bitcoin - profit;
outcomes.push((
strike as i64,
strike_ctx
.derive_num(strike as u64)?
.template()
.add_output(profit, &v.user_api.receive_payment(profit), None)?
.add_output(refund, &v.operator_api.receive_payment(refund), None)?
.into(),
));
}
// Now that the schedule is constructed, build a contract
Ok(GenericBetArguments {
// must send max amount for the contract to be valid!
amount: max_amount_bitcoin,
outcomes,
oracle: v.operator_api.get_oracle(),
cooperate: Clause::And(vec![key, user]),
symbol: v.symbol,
})
}
}
impl<'a> TryFrom<RiskReversal<'a>> for GenericBet {
type Error = CompilationError;
fn try_from(v: RiskReversal<'a>) -> Result<Self, Self::Error> {
Ok(GenericBetArguments::try_from(v)?.into())
}
}
Woooooop! Our Risk has been Reversed!
Options
First let’s play with Options.
We need a generic trait interface for all options. The basics
are something to happen when it expires, and something to happen
when it is paid for (strikes).
/// Generic functionality required for Expiring contracts
pub trait Expires: 'static + Sized {
decl_then! {
/// What to do when the timeout expires
expires
}
decl_then! {
/// what to do when the holder wishes to strike
strikes
}
}
First we’ll define an ExpiringOption whereby two parties deposit
all the funds required for the contract (full collateral).
/// Wraps a generic option opt with functionality to refund both parties on timeout.
pub struct ExpiringOption<T: 'static> {
party_one: Amount,
party_two: Amount,
key_p1: bitcoin::Address,
key_p2: bitcoin::Address,
key_p2_pk: Clause,
opt: T,
timeout: AnyAbsTimeLock,
}
impl<T> Contract for ExpiringOption<T>
where
GenericBet: TryFrom<T, Error = CompilationError>,
T: Clone + 'static,
{
declare!(then, Self::expires, Self::strikes);
declare!(non updatable);
}
impl<T> ExpiringOption<T> {
/// Party Two is the option holder
#[guard]
fn signed(self, _ctx: Context) {
self.key_p2_pk.clone()
}
}
Then we’ll implement the functions:
impl<T> Expires for ExpiringOption<T>
where
GenericBet: TryFrom<T, Error = CompilationError>,
T: Clone,
{
#[then]
fn expires(self, ctx: sapio::Context) {
// return the money to each party
ctx.template()
.add_output(
self.party_one.into(),
&Compiled::from_address(self.key_p1.clone(), None),
None,
)?
.add_output(
self.party_two.into(),
&Compiled::from_address(self.key_p2.clone(), None),
None,
)?
.set_lock_time(self.timeout)?
.into()
}
/// Only party 2 can strike!
#[then(guarded_by = "[Self::signed]")]
fn strikes(self, ctx: sapio::Context) {
// Send the money to a generic bet...
ctx.template()
.add_output(
(self.party_one + self.party_two).into(),
&GenericBet::try_from(self.opt.clone())?,
None,
)?
.into()
}
}
Now we’ll implement similar logic, but where the amount from party_two
is not paid until the strike is called:
/// Similar to `ExpiringOption` except that the option requires an additional
/// value amount to be paid in in order to execute, hence being "under funded"
pub struct UnderFundedExpiringOption<T: 'static> {
party_one: Amount,
party_two: Amount,
key_p1: bitcoin::Address,
opt: T,
timeout: AnyAbsTimeLock,
}
impl<T> Contract for UnderFundedExpiringOption<T>
where
GenericBet: TryFrom<T, Error = CompilationError>,
T: Clone + 'static,
{
declare!(then, Self::expires, Self::strikes);
declare!(non updatable);
}
impl<T> Expires for UnderFundedExpiringOption<T>
where
GenericBet: TryFrom<T, Error = CompilationError>,
T: Clone,
{
#[then]
fn expires(self, ctx: sapio::Context) {
ctx.template()
.add_output(
self.party_one.into(),
&Compiled::from_address(self.key_p1.clone(), None),
None,
)?
.set_lock_time(self.timeout)?
.into()
}
#[then]
fn strikes(self, ctx: sapio::Context) {
ctx.template()
.add_amount(self.party_two)
.add_sequence()
.add_output(
(self.party_one + self.party_two).into(),
&GenericBet::try_from(self.opt.clone())?,
None,
)?
.into()
}
}
NFT Exercises for the reader:
- Question: Why isn’t a normal NFT Sale contract an Option?
- Answer: Because it doesn’t guarantee the uniqueness of the right to purchase
- Question: How can we implement NFT Options?
- Answer: The NFT Option has to be the only owner of the NFT. Without writing any new contracts… generate the code to transfer the NFT to a 2-2 multisig between option holder and seller, and pre-sign a timelocked transfer back to the original owner plus a non-timelocked sale to the purchaser for a price. With new contracts? Do the same thing, but without having to stitch it together.
- Question: Can you do call options? What about put options?
- Answer: Sure! For a call option have the contract have the NFT in it. For a
put option, require that the NFT be put in and the funds present, and pre-sign
the transfer. The tricky thing is that if you wish to ‘move’ your NFT while you
have a put option open, you must get your counterparty to agree to the new UTXO
representing the NFT. But they can validate this client side and sign automatically.
Does this NEED CTV?
No, not in particular. Most of this stuff could be done with online signer server federation between you and counterparty. CTV makes some stuff nicer though, and opens up new possibilities for opening these contracts unilaterally.
Representing Positions as NFTs
Offers to open up a contract could be represented as NFTs! You don’t even need
to create the NFT, just bind the NFT interface with an option open as a generic
minting parameter, and then you can do price discovery of Option contracts through a dutch auction you thought was just for selling cat pics.
Wen LN?
Well, if you note that we can coop close options and derivatives, and that I
claimed we don’t need CTV, these two facts imply that you can put these kinds of
contracts inside of the LN no problem :).
What About PowSwap?
I mentioned powswap.com earlier. But you’ll have to wait
to read about it, that’s all for today!