Finished the first draft of the new paper.

doc/BlocktreeCloudPaper/BlocktreeCloudPaper.tex

@@ -158,18 +158,24 @@ which is the primary use case this method was intended for.
 Awaiting replies to messages serves as a simple way to synchronize a distributed computation.
 
 % Description of virtual actor system.
-One of the challenges when building actor systems is supervising and managing actor's lifecycles.
+One of the challenges when building actor systems is supervising and managing actors' lifecycles.
 This is handled in Erlang through the use of supervision trees,
 but Blocktree takes a different approach inspired by Microsoft's Orleans framework.
 Orleans introduced the concept of virtual actors,
 which are purely logical entities that exist perpetually.
-In Orleans, one does not need to spawn actors nor worry about respawing them should they crash,
+In Orleans, one does not need to spawn actors nor worry about respawning them should they crash,
 the framework takes care of spawning an actor when a message is dispatched to it.
 This model also gives the framework the flexibility to deactivate actors when they are idle
 and to load balance actors across different computers.
 In Blocktree a similar system is used when messages are dispatched to services.
 The Blocktree runtime takes care of routing these messages to the appropriate actors,
 spawning them if needed.
+A service must be registered in a runtime before messages can be routed to it.
+The actors which are spawned based on this registration are called \emph{service providers} of the
+service.
+Services which directly use operating system resource,
+such as those that listen on network sockets,
+are often started immediately after registration so that they are available to external clients.
 
 % Message addressing modes.
 Messages can be addressed to services or specific actors.
@@ -183,7 +189,7 @@ which contains the following fields:
 \begin{enumerate}
   \item \texttt{service}: The path identifying the receiving service.
   \item \texttt{scope}: A filesystem path used to specify the intended recipient.
-  \item \texttt{rootwards}: An boolean describing whether message delivery is attempted towards or
+  \item \texttt{rootward}: A boolean describing whether message delivery is attempted towards or
     away from the root of the filesystem tree. A value of
     \texttt{false} indicates that the message is intended for a runtime directly contained in the
     scope. A value of \texttt{true} indicates that the message is intended for a runtime contained
@@ -308,6 +314,8 @@ the child receives a message when the parent returns.
 Service providers spawned by the runtime are owned by it.
 They continue running until the runtime chooses to reclaim their resources,
 which can happen because they are idle or the runtime is overloaded.
+Note that ownership is not limited to a single runtime,
+so distributed resources can be managed by owning actors in many different runtimes.
 
 % Message routing to services.
 A service is identified by a Blocktree path.
@@ -318,8 +326,6 @@ The runtime spawns a new actor when it finds no service provider associated with
 message it is delivering.
 Some services may only have one service provider in a given runtime,
 as is the case for the sector and filesystem services.
-Services are reactive,
-they don't do anything until they receive a message to process.
 The \texttt{scope} and \texttt{rootward} field in an actor name specify the set of runtimes to
 which a message may be delivered.
 They allow the sender to express their intended recipient,
@@ -781,8 +787,8 @@ using Blocktree,
 but there is an entire world of existing applications which only know how to access the local
 filesystem.
 To allow these applications access to Blocktree,
-a FUSE daemon is included which allows a Blocktree directory to be mounted to a directory in the
-local filesystem.
+a FUSE daemon called \texttt{btfuse} is included which allows a Blocktree directory to be mounted
+to a directory in the local filesystem.
 This daemon can directly access the sector files in a local directory,
 or it can connect over the network to filesystem or sector service provider.
 This FUSE daemon could be included in a system's initrd to allow it to mount its root filesystem
@@ -1218,13 +1224,15 @@ However, Postgres does have a container image available on docker hub,
 we can create a service to run this container image in our domain.
 But Postgres stores all of its data in the local filesystem of the machine it runs on.
 How can we ensure this does not become a single point of failure?
-First, we should create a directory in our domain to hold the Postgres cluster.
+First, we should create a directory in our domain to hold the Postgres cluster directory.
 Then we should procure at least three servers for our storage cluster
 and provision runtimes hosted on each of them in this directory.
 The sector service is registered on each of the runtimes,
 so all the data stored in the directory will be replicated on each of the server.
 Now, the Postgres service should be register in one and only one of these runtimes,
 as Postgres requires exclusive access to its database cluster.
+\texttt{btfuse} will be used to mount the Postgres directory to a path in the local filesystem
+and the Postgres container will be configured to access it.
 We now have to decide how other parts of the system are going to communicate with Postgres.
 We could have the Postgres service setup port forwarding for the container,
 so that ordinary network connection can be used to talk to it.
@@ -1263,19 +1271,219 @@ the new service will come up after a brief delay
 and the system will recover.
 
 \subsection{A realtime geo-spacial environment.}
-% Explain my vision of the metaverse.
-
+% Motivation
+If we are to believe science fiction,
+then the natural evolution of computer interaction is the development
+of a persistent virtual world that we use to communicate, conduct business, and
+enjoy our leisure.
+This kind of system has been a dream for a long time,
+but as it has grown closer to becoming a reality,
+the popular consciousness has shifted against it.
+People are rightly horrified by the idea of giving control over their virtual worlds to the same
+social media company which has an established track record for causing societal harm.
+But this technology does not need to be dystopian.
+If an open system can be built, which actually works,
+it can prevent the market from accepting a closed system designed to lock in user attention
+and monetize them relentlessly.
+This is the future,
+it is only a question of who will own it.
+
+% Coordinates
+Let us explore how Blocktree could be used to build such a system.
+The world we are going to render will be a planet with a roughly spherical surface and a
+configurable radius $\rho$.
+$\rho$ is a \texttt{u32} value whose units are meters.
+We will use latitude ($\phi$) and longitude ($\lambda$) in radians to describe the locations of
+points on the surface.
+Both $\phi$ and $\lambda$ will take \texttt{f64} values.
+The elevation of a point will be given by $h$,
+which is the deviation from $\rho$.
+$h$ is measured in meters and takes values in \texttt{i32}.
+So, the distance from the center of the planet to the point ($\phi$, $\lambda$, $h$) is
+$\rho + h$.
+
+% Directory organization. Quad-trees.
+The data describing how to render a planet consists of its terrain mesh, terrain textures, and
+the objects on its surface.
+This could represent a very large amount of data for a planet with realistic terrain populated by
+many structures.
+To facilitate sharding the information in a planet over many different servers,
+the planet is broken into disjoint regions,
+each of which is stored in its own directory.
+A single top-level directory represents the entire planet,
+and contains a manifest describing it.
+This manifest specifies the planet's name, its radius, its rotational period,
+the size of its regions in MB, as well as any
+other global attributes.
+This top-level directory also contains the texture for the sky box to render the view of
+space from the planet.
+In the future it may be interesting to explore the creation of more dynamic environments surrounding
+the planet,
+but a simple sky box has the advantage of being efficient.
+The data in a planet is recursively broken into the fewest number of regions such that the
+amount of data in each regions is less than a configured threshold.
+When a regions grows too large it is broken into four new regions by cutting it along the
+centerline parallel to the $\phi$ axis, and the one parallel to the $\lambda$ axis.
+In other words, it is divided in half north to south and east to west.
+The four new regions are stored in four subdirectories of the original region's directory
+named 0, 1, 2, and 3.
+The data in the old region is then moved into the appropriate directory based on its location.
+Thus the directory tree of a planet essentially forms a quad-tree,
+albeit one which is built up progressively.
+
+% Region data files.
+In the leaf directories of this tree the actual data for a region are stored in two files,
+one which describes the terrain and the other which describes objects.
+It is expected that the terrain will rarely be modified,
+but that the objects may change regularly.
+The terrain file contains the mesh vertices in the region as well as its textures.
+It is organized as an R-tree to allow for efficient spacial queries based on player location.
+The region's objects file is also organized as an R-tree.
+It contains all of the graphical data for the objects to be rendered in the region,
+such as meshes, textures, and shaders.
+
+% Plots.
+The creation of a shared virtual world must involve players collaboratively building persistent
+structures.
+This is allowed in a controlled way by defining plot objects.
+A plot is like a symbolic link,
+it points to a file whose contents contain the data used to render the plot.
+This mechanisms allows the owner of the planet to delegate a specific area on the surface
+to another player by creating a plot defining that area and pointing it to a file owned by the
+player.
+The other player can then write meshes, textures, and shaders into this file to describe the
+contents of the plot.
+If the other player wishes to collaborate with others on the construction,
+they can grant write access on the file to a third party.
+This is not unlike the ownership of land in the real world.
+
+% LOD files in interior directories.
+To facilitate the viewing of the planet from many distances,
+each interior node in the planet's directory tree contains a reduced level of detail (LOD) version
+of the terrain contained in it.
+For example, the top-level directory contains the lowest LOD mesh and textures for the terrain.
+This LOD would be suitable for rendering the planet as a globe on a shelf,
+or as it would appear from a high orbit.
+By traversing the directory tree,
+the LOD can be increased as the player travels closer to the surface.
+This system assist with rendering an animation where the player appears to approach and land upon
+the planet's surface.
+
+% Sharding planet data.
+By dividing the planet's data into different leaf directories,
+it becomes possible to provision computers running the sector service in each of them.
+This divides the storage and bandwidth requirements for serving the planet over this set of
+servers.
+In addition to serving these data,
+another service is needed to keep track of player positions and execute game logic.
+Game clients address their messages using the directory of the region their player is located
+in, and set \texttt{rootward} to true.
+These messages are delivered to the closest game server to the region the player is in,
+which may be located in the region's directory or higher up the tree.
+When a player transitions from one region to the next,
+its game client begins addressing messages using the path of the next region as the scope.
 
 
 \section{Conclusion}
-% Blocktree serves as the basis for building a cloud-level distributed operating system.
+% Blocktree serves as the basis for building distributed Unix.
+There have been many attempts to create a distributed Unix over the years.
+Time has shown that this is a very hard problem,
+but time has not diminished its importance.
+IT systems are more complex now than ever,
+with many layers of abstraction which have built up over time.
+We have suffered greatly from systems which were never designed to be secure on the hostile internet
+that exists today.
+Security has been bolted onto these systems (HTTPS, STARTTLS, DNSSEC)
+in a backwards compatible way,
+which results in weakened protections for these systems.
+What's worse,
+the entire trust model of the web relies on the ludicrous idea that there is a distinguished group
+of certificate authorities who have the power to secure our communications.
+We need to take a different approach.
+Data should be certified by its path,
+it must always be transported between processes in an authenticated manner,
+and user code should never have to care how this is accomplished!
+Time will tell whether the programming model of Blocktree is comprehensible and useful for
+developers,
+but the goal is to create the kind of easy to extend computing environment which allowed Unix to
+be successful.
 
 % The system enables individuals to self-host the services they rely on.
-
-% It also gives business a freeer choice of whether to own or lease computing resources.
-
-% The system advances the status quo in secure computing.
+These days, the typical internet user stores all of their important data in the cloud with
+third-party service providers.
+They do this because of the convenience of being able to access this information from anywhere,
+and because of the perceived safety in having a large internet company look after it for them.
+This convenience comes at the price of putting users at the mercy of these companies.
+Take email for example,
+a service which is universally used for account recovery and password reset.
+If a service provided decided to stop providing a user access to their email,
+the user would be effectively cut off from any website which sends login verification emails.
+This is not a hypothetical situation,
+such an incident has occurred (TODO: INSERT CITATION FROM LVL1).
+There is no technical reason for things to be this way.
+Blocktree allows users to host their own services in their own domain.
+If we can make setting up an email or VOIP server as simple as clicking a button in a web GUI,
+their will be no convenience advantage to cloud services.
+One challenge for self-hosting data is ensuring it is protected from loss when hardware inevitably
+fails.
+The data redundancy in Blocktree's sector layer ensures that the loss of any one storage
+device will not cause data loss.
+Streaming replication can also be used to maintain additional redundant copies.
+If more users begin hosting their own services,
+the internet will become more distributed,
+which will make it more resistent to disruption and centralized control.
+
+% Benefits to businesses.
+Cloud computing has also driven changes to the way businesses acquire computing resources.
+It is common for startups to rent all of their computing resources from one large cloud
+provider.
+There are compelling economic and technical reasons to do this,
+but as a firm grows they often experience growing pains as their cloud bills also grow.
+If the firm has not developed their software with a multi-cloud, or hybrid approach in mind,
+they may face the prospect of major changes in order to bring their application on-prem or to a
+rival cloud.
+By developing their application on Blocktree,
+businesses have a single platform to target which can run on rented computers in the cloud just as
+easily servers in their own data center.
+This ensures the choice to rent or buy can be made on a purely economic basis.
+Blocktree is not the only system that provides this flexibility.
+The portability of containers is one of the reasons they have become so popular.
+Containers have their place and will most likely be used for years to come,
+but they are a lower level abstraction which requires the developer to the problems that Blocktree
+handles.
+
+% Blocktree advances the status quo in secure computing.
+Ransomware attacks and data breaches are embarrassingly common these days.
+There are many reasons for this,
+from the reliance on passwords for authentication, to the complexity of the software supply chain,
+but it is clear that as IT professionals we need to do more to keep the systems under our
+protection safe.
+Blocktree helps to do this by solving many of the difficult problems involved with securing
+communication on a hostile network.
+It takes a true zero-trust approach,
+ensuring that all communications between processes is authenticated using public key cryptography.
+Data at rest is also secured with encryption and integrity protection.
+No security system can prevent all attacks,
+but by putting these mechanisms together in an easy to use platform,
+we can advance the status quo of secure computing.
 
 % Composability leads to emergent benefits.
+When Unix was first developed in the 1970's, its authors could not have foreseen the applications
+that would be enabled by their system.
+Although there have been many different kinds of Unices over the years,
+the core programming model, built around the filesystem, has remained since the beginning.
+It is a testament to the importance of this abstraction that it has persisted for so long.
+No designer can foresee all the ways that their abstractions will be used,
+but they can try to build them in such a way that as much choice is left to the user as possible.
+By making the actor model, and messaging passing, the core of Blocktree,
+it is hoped that low overhead communication between distributed components can be achieved.
+By using this system to provide a global distributed filesystem,
+it is hoped that the interoperable sharing of data can be achieved.
+And by using protocol contracts to constrain actor communication,
+it is hoped that the structure and safety of type theory can bring order to distributed
+computation.
+While it is possible to see some of the applications that can be built from these abstractions,
+it seems likely that their composability and the creativity of developers will enable systems that
+cannot be foreseen.
 
 \end{document}