#open source

I’m pretty bullish on Bitcoin so a few months ago I set out to build a “pure” Bitcoin related application. Specifically, I was looking to build an application that leveraged the Bitcoin network directly, without using any third party APIs or services. The goal behind avoiding third party services was to explore how difficult using the Bitcoin network directly is and also to embrace Bitcoin’s decentralized nature and not rely on another company to move coins.

Conceptually the way the Bitcoin network works is relatively straightforward. You move coins by creating transactions which are just messages written and cryptographically signed in a specific format and then you listen for transactions which include your addresses to keep your balance up to date. Of course, the devil is in the details and there’s a dauntingly large number of them. For example, Ken Shiriff explains how to craft a transaction by hand in Bitcoins the hard way: Using the raw Bitcoin protocol and it’s no easy read. Given that just crafting transactions involved so much code, I started researching existing open source libraries that facilitate working with Bitcoin.

After doing some research, it looked like the most popular approach to interfacing with the network directly was to run the bitcoind deamon and then make RPC calls to the exposed functions. Objectively, using RPC calls to bitcoind qualifies as a “pure” solution but I still didn’t love it. After a bit more searching, I came across bitcoinj which is a pure Java library for working with Bitcoin.

Unlike bitcoind, bitcoinj is a library so its designed to be embedded in other codebases and it supports simplified payment verification (SPV) which allows it to operate without downloading the entire blockchain, ~25GB as of today. On top of this, its written in Java so it’s easy to use from Scala, something I’d been looking to experiment with.

So what are some key takeaways from using bitcoinj?

  • Test on the TestNet: Although debugging new code with live money would be thrilling. the TestNet allows you to develop against the Bitcoin network without using “live” coins. With bitcoinj, if you instantiate TestNet3Params you’ll connect into the Testnet.
  • TestNet Fountains: Testing is pretty tough without some coins but luckily you can use fountains to grab some. http://faucet.xeno-genesis.com/ worked well for me to grab some coins. Just a heads up, TestNet transactions take awhile to get broadcast and included into the Blockchain so be patient.
  • Extend WalletAppKit: WalletAppKit is a utility class which helps bootstrap everything you need for a SPV wallet and will save you some headaches. I started off trying to hand roll everything and quickly switched over to extending WalletAppKit.
  • Chain API: Since the Chain API can be used on the TestNet it’s useful for testing your own applications. Specifically, Bitcoinj treats sending and receiving coins to “yourself” weirdly so it’s more effective to use Chain to test transactions.
  • Coin.parseCoin: Sorting out how many Satoshis you want to send is surprisingly difficult since sendCoins accepts a Coin object. Luckily, Coin.parseCoin will accept a BTC value and return representative Coin object.
  • Blockchain downloads and wallet start up slow down development: Even in SPV mode, it still takes a non-trivial amount of time to “catch up” if your client hasn’t been running for awhile. On top of that, starting up the wallet takes a few minutes since it has to discover and connect to beers. What this translates to is resuming development after being away for a while is slow and the “edit, compile, test” loop isn’t as tight as usual.
  • It’s real money: This should go without saying but once you’re running on the live Bitcoin network you’re playing with real money so be careful. If your keys get out anyone will be able to move your coins out of your addresses. So be careful what you check in on git and if you’re building something public definitely take a second to lock down your server.

Anyway, this was my first time building something Bitcoin related and it was a positive experience. The project is still private but I’ll definitely share it once it’s released. As always, questions or comments are welcome!

Posted In: Bitcoin

Tags: , , ,

Last week we were looking to leverage a set of JSON API endpoints in a Symfony2 project to power a single page Javascript app. The way the API had been setup, the routes were all secured with an HTTP Basic Auth firewall matching on “/api”. This worked great for the mobile apps but for a Javascript app it would be awkward to have the user re-enter their credentials to authorize the basic auth firewall. What we really wanted to do was to leave everything “as is” but have Symfony use the normal cookie based firewall when we passed in a special “isOnlineApp” parameters on the URL.

Unfortunately, setting something like this up with the default “pattern” setting in your security.yml file isn’t possible. The “pattern” setting only matches on the route URL, not the parameters so there’s no way to have it selectively trigger when a parameter is present on a URL. So how do you do it? Well as it turns out, there’s a firewall configuration called “reuqest_matcher” which lets you “match” a firewall using a service. Just create a service that extends the RequestMatcherInterface, implment a “matches” function, and then add the class as a service.

Our code for the service ended up looking like:

And then the actual firewall configuration ends up being:

You don’t need a “pattern” setting anymore since the “matches” function supersedes it. Anyway, let me know if you have any questions!

Posted In: PHP, Symfony

Tags: , ,

At the beginning of the summer we decided to redo our website. The design on the old site was looking a bit dated and more importantly the content didn’t really reflect the types of projects we’re looking to work on. From a technology perspective, our old site was built on WordPress with the explicit goal of being able to share the same WordPress theme as our blog. The two sites did in fact share the same theme but looking back, we never updated the main site to really make it “worth it”. With that experience in mind, we started looking around for what we could use to build setfive.com.

Stepping back and looking at our requirements, we really don’t need a CMS. I’d argue this holds true for most website projects when there’s less than 20 pages, everyone who might edit it is technical, and the content isn’t updated frequently. Specifically looking at some major WordPress features, we don’t need the WYSIWYG editor, plugin ecosystem, media handling, or theming capabilities. So what capabilities do we need?

  • Routing / Pretty URLs:
    Serving “raw” URLs like “about.php” hasn’t been OK since IE6 so this is obviously a “must have”. Basically, we need some way to map human readable URLs to specific pieces of content. Sure, you could do all of this with mod_rewrite or nginx’s location directive but that just sounds awkward. Whatever we pick should be able to handle these translations internally.
  • A modern templating engine:
    PHP “grew up” as a web templating system but today it’s missing some key features in comparison to purpose built templating engines. Being handicapped by only having “require” or “include” and being forced to set global variables in 2014 is terrible so “real” templating should be a requirement.
  • Access to a CGI/scripting language:
    I’m sure there’s “razor” of some sort describing a phenomenon where eventually a “static” website will need access to dynamic capabilities. For us, from the outset we knew we’d need to do things like pull in RSS feeds and send contact emails so we knew we needed access to some sort of CGI.

There’s certainly more capabilities static websites could need but I think this is a decent list for the “general” case and it captures our requirements. After doing some research, it looks like there’s currently a few options that would satisfy these requirements:

  • Static site generators: Generally, products like Jekyll or Phrozn will “compile” a set of your templates into static HTML pages which would then just be served by Apache or nginx. These solutions are fine except that they’d make things like integrating a RSS feed awkward.
  • “Lightweight” CMS: There’s a slew of “lighter” WordPress alternatives like Apostrophe or Ghost that check all the boxes but come with their own issues. With any “platform” you’d be back to learning the quirks of another platform and of course keeping the software up to date.
  • Use a micro-framework: Frameworks like Sinatra or Silex would offer the templating and routing features found in a “full framework” without the associated code footprint. In my opinion, this option offers the best combination of familiarity, extensibility, and a small enough footprint that updates shouldn’t be a huge issue.

I ultimately chose Silex because our team has deep PHP experience, especially with Symfony2. Because of that we’d be right at home with the Routing component and of course Twig for templating.

OK so how do you actually get this to work? I ran across Jonathan Petitcolas’s Building a static website with Silex post and used it as a guide. Here are the actual commands you’d need to get this all setup though:

Now, you just need to create a file named “index.php” which contains:

And finally, in the “views” directory add a file called “index.html.twig” which contains some content. If you have a web server setup, just point a vhost at the “web” directory, load it, and you should see the content of your index file.

If you don’t have a web server setup, a nifty trick via Gonzalo Ayuso, create a in the “web” directory named “router.php” containing:

And now, you can start the built in PHP 5.4+ server by running:

You can load your Silex app by loading http://localhost:8888 in your browser.

Anyway, as always questions and comments are welcome!

Posted In: PHP

Tags: , ,

I was building out an API test console a few days ago and realized I’d never actually looked into how to grab all available routes in Symfony2. The “console” is basically a form with a select box and textarea that lets you “ping” the REST API routes in one of our applications. To make this work, I wanted to traverse all the registered routes, filter for the ones that contained “api_”, and then generate dummy URLs for those routes.

I searched around a bit for how to grab all the registered routes and the only link seems to be https://gist.github.com/hubgit/3380250 Unfortunately, if you try and use the code you’ll discover that “getPattern” no longer exists in the CompiledRoute class. It looks like it’s been replaced by getPathVariables

So, working code to generate a list of route names and “dummy” URLs for you API routes ends up looking like:

Posted In: PHP, Symfony

Tags: ,

One of the nicest features of Symfony2 is the Request/Response paradigm for processing a HTTP request and then sending a response back to a client. At a high level, Symfony’s HttpFoundation component provides an object oriented abstraction to easily deal with HTTP requests and generate responses to send back to a client. Assuming application code correctly uses HttpFoundation, it will only interact with request variables through the Request class, as opposed to $_REQUEST, and only send output using the Response class, as opposed to an “echo”. Because of this contract, the framework as a whole makes it easy to manipulate responses before they’re sent back to a client.

A typical use case that leverages this would be logging API responses before they’re sent back to a client. As much as an API might be RESTful, at some point it’s easier to debug things when you can see the responses that clients have been receiving. OK great so how do you do it? It’s actually pretty straightforward, just create a class to receive the “kernel.terminate” event and register it as a service with the appropriate tags:

And then create the class where you want to manipulate or log the requests:

And that’s about it!

Note: Per Andras’ comment below the event has been switched to “kernel.terminate”.

Posted In: Symfony

Tags: , ,