Code Description



Most of the fun data storage stuff is in: org.rlcommunity.bt.recordbook.dataStorage



The Main Idea

The current idea for how data is stored in bt-Trainer (which will serve as the starting point for bt-RecordBook) is as follows.

Each time a computer starts running an experiment, it creates 4 files: Two index files and two results files.  The pairs of files are labeled a and b.  For example, after doing one run, the data directory might look like:
-rw-r--r--  1 btanner  btanner    80K  4 Jun 12:26 results_1212603842322_a.data
-rw-r--r--  1 btanner  btanner   152K  4 Jun 12:26 results_1212603842322_a.index
-rw-r--r--  1 btanner  btanner    80K  4 Jun 12:26 results_1212603842322_b.data
-rw-r--r--  1 btanner  btanner   152K  4 Jun 12:26 results_1212603842322_b.index

The second index and results files are duplicates for redundancy.  When writing data, the program first writes the 'a' files and then the 'b' files, so if it is terminate unexpectedly,  there is no chance that both 'a' and 'b' files are corrupted.  I have run into many problems with corrupt files in my Java experiments historically, which is why I'm doing this.  I tried to find a more elegant approach that was truly robust, and had very little luck.  I'm open to suggestions.

If another experiment is started, 4 more files will be created.  Or, if this experiment is stopped and restarted, 4 more files.  Basically, these 4 files belong to one particular execution of the experiment, and they are never written to again.  This reduces the chance that they will ever be corrupted by anyone else.  The files are named with the current time in milliseconds since the epoch to give them each a unique name.  We could maybe think of something a little more clever to reduce the unlikely but eventual chance of a naming collision.

In practice, we want to "clean" the directory and see remove any redundant or corrupt 'a' or 'b' indices and data.

The Index File

The index file describes what experimental parameter runs exist in the data file.  Each entry in the index file describes an experiment:
    String envName;
    String agentName;
    protected String pEnv;
    protected String pAgent;
    String paramSummary;

envName and agentName should be select explanatory.  pEnv and pAgent are string serialized versions of the parameterholders that were used to configure the agent and environment.  These are longish.

paramSummary is a shorter string that encapsulates both pEnv and pAgent.  The Param Summary is created by method getParamSummaryStringAndSetParams in org.rlcommunity.bt.recordbook.experimenter.AbstractExperiment.java

Important Note: We use some maps and sets to keep track of different run configurations.  Each run configuration is indexed by the ResultRecord.getExperimentKey().  All this is doing i staking paramSummary.hashcode(), which returns an int.

TODO: Should write some sanity code somewhere just to make sure that we don't have any hashcode collisions.  Dan Lizotte suggests using a MD5 hash instead of String.hashcode() and use the Java MessageDigest code to do it.

The Data File

The data file is a big list of AbstractRunRecords, where each ResultRecord is a list of AbstractRunRecords.  Wen the AbstractRunRecords are written to the file, they write the abstract stuff that doesn't change, and then they call writeDataPayload on the specific concrete class that they are to write the specific type of data out.


So for example, with and EpisodeEndPointRunRecord, AbstractRunRecord first writes:
        out.writeInt(getType().id());
        out.writeInt(experimentKey);
        out.writeObject(runKey);
        out.writeLong(runTimeInSeconds);
        out.writeObject(runDate);

And then, the EpisodeEndPointRunRecord will add:
        out.writeObject(episodeEndPoints);

which is an array of ints.


Reading in a saved index file

As we read in the index file we build a temporary Map, which maps the ExperimentKey to the ResultRecords.

Then, we read in the RunRecords from the data file one at a time, we append them to the appropriate ResultRecords.

The reason this is good is that when we want to add a result, we just need to read the index file (perhaps add a new entry to the index) and then drop the result at the bottom of the data file.

We're also going to give each record a unique ID so that if we read the same record in from 3 files, we only use it once.

Java Class Descriptions


AbstractRunRecord

Main Data

    private long runTimeInSeconds;
    private Date runDate;
    private final int experimentKey;
    private final UUID runKey;


The AbstractRunRecord is at the top of the hierarchy of what we'd want to capture about any "run", including when it happened and how long it took.  The experimentKey links it to a particular experiment, and the runKey is something that is unique within (the experiment?) so we can store references to it in summaries and stuff and always be able to call it up.


episodicRunRecordInterface

This is an interface you should implement if you want to the record to be interpretable easily in terms of "how many episodes were completed in some amount of time".

EpisodeEndPointRunRecord extends AbstractRunRecord implements episodicRunRecordInterface

Main Data

int[] episodeEndPoints = null;

The EpisodeEndPointRunRecord is the record that is stored for a run in a problem where you care about how many episodes have been completed at various points in time.

Proposed changes:
  • Endpoints should be longs instead of ints.  One say someone will run an experiment longer than 2 billion time steps and using an int will cause a problem

EpisodeEndReturnRunRecord extends AbstractRunRecord implements implements returnRunRecordInterface

Main Data

double[] episodeReturns = null;

The EpisodeEndReturnRunRecord is the record that is stored for a run in a problem where you care about much reward the agent gathered in each episode.


Comments