It will be created by Brian Tanner (and company) at the University of Alberta (please tell us if you want to help).
The interface for RL communication will be based on RL-Glue.
Agents and environments will come from the RL-Library, with usability extensions (for exposing parameters) from RL-VizLib.
The initial version of the record book will be based on Java environments and agents in order to get us moving quickly.
The experimental data will be stored (in a format yet to be discussed) in Amazon's Simple Storage System.
Experiments can be verified and validated on any computer, but all official runs will be done remotely on virtualized hardware provided by Amazon's Elastic Compute Cloud. For now, this will be paid for by Brian Tanner's personal research budget.
a new project that allows a remote system to do the authorization, and patched open source projects Typica (SQS + SimpleDB) and Jets3t (S3) to use this service.
So now, I can distribute the software to people, and by logging with with username and password I assign, they can access these services without me compromising my secret key.
The plan now is that we will put the software to do bt-recordbook computation on all sorts of computers around the lab and run them at times of low use (overnight and away from paper deadlines). We will also invite the reinforcement learning community to do the same. Basically, any computer with Java 1.5 or higher and an Internet connection can be adding results to the recordbook.
I believe we can get hundreds, maybe even thousands machines working at the same time generating RL results to fill in the recordbook and make it a game-changing sources of RL results.
But why would anyone want to go through the hassle of creating an account and running the software on their computer? Credits. Each participant will earn recordbook credit for running these experiments. The exact economy hasn't been worked out yet, but the basic idea is that for every K hours of computing you do, you early 1 hour of computing credits. Credits can then be exchanged for time on other people's computers -- having your experimental results computed using the recordbook "grid". Credits can be spent on public results (common good), or on private results.
We don't all have the same deadlines, so credits can be used at "crunch time" to run a large number of experiments when putting together a paper. We can also use scalable computing resources like the elastic compute cloud to ensure that we get high throughput even at big paper deadlines that we share. With a financial support system (I'm working on it), we can afford to spawn hundreds of virtual machines in the cloud to pick up the slack when everyone tries to spend their credits at a major paper deadline. This can be partially supported by making it possible to buy credits at a small premium.
For example, say you need about 400 hours of computation done within 12 hours and you don't have machines to do the work for you. If our price was .25 cents per credit-hour, then you could spend $100 and have the computation done immediately by the grid, within the 12 hours. If the grid is loosely loaded, then your work is done by others who are earning credit, and the $100 goes into the bank to pay for virtual nodes in the cloud for times when the grid is heavily loaded. If the grid is heavily loaded right now, then the money is spent directly on virtual nodes in the cloud to run your experiments. You you don't need to buy, configure, or manage the machines or infrastructure to run and coalesce these experimental results! You pay a reasonable price for your computing time, and you get your results: fast! Better yet, if you've been saving credits by doing work for other people in the past, it's free!
The only thing stopping this from working is initial capital. We need some money to get started with such a system, if we want to be able to expand to the cloud at deadline time. I think the strategy for this will be for me and/or my colleages/supervisor etc to "pay" into the account for having our own experiments and experiments for the community's good at first. We would want to run these experiments anyway, and don't have enough computers to run them, so we can pay for the initial credits that people are earning. Once there is a couple thousand dollars set aside, we can see how things go at the first major paper deadline. Even $1000 should actually go pretty far.
The infrastructure for this system is all in place. I don't have a user signup yet, but I can manually add users, and then they can run experiments and start earning credit. I also don't have an automated way for users to submit their own experiments to run yet, or for experiments to be private, but I'm working on it.
This is the current model of the recordbook. Anyone with a recordbook account can submit and run experiments, and the results are all aggregated on virtual machines in the amazon compute cloud. Currently only public results are available, this will be extended to allow for private results. Also. Private results that were created in cloud mode will be publishable directly into the recordbook. Results created in Cluster or Single Computer mode will not.
Strikethrough means it is "cancelled", green means it's done.
Some of these things are listed as official "issues" already, some are not