Преглед изворни кода

* Factored the serialization code out of the node crate, as I
realized I'll need to use it in multiple binaries.
* Wrote out a broad-strokes roadmap.

Matthew Carr пре 3 година
родитељ
комит
5851021348

+ 96 - 0
README.md

@@ -1,3 +1,99 @@
+### Project Road Map
+In broad strokes, theses are the phases of development.
+
+## Phase 1: Foundations
+* Definition of the main data types.
+* Definition of the binary format used to serialize all data structures.
+* Creation of a cryptographic abstraction layer which will defer to openssl for the time being.
+This module will include type definitions for hashes, keys and signatures. It will define
+functions for:
+  1. Symmetric encryption and decryption.
+  2. Asymmetric encryption and decryption.
+  3. Asymmetric signing and verification.
+  4. Higher level functions for operating on blocks, read caps and write caps.
+* Implementation of a block manager for persisting blocks to disk and retrieving them when needed.
+We need to make sure the page cache is being used while doing IO here.
+* Implementation of a cache for block keys.
+* Definition of an RPC interface for sending commands to and receiving responses from a node via
+  stdin. 
+* Creation of a crate called `harness` for running nodes as child processes and communicating with
+them. This crate must define high level functions for:
+  1. Preparing a directory for a new node.
+  2. Starting a node.
+  3. Sending commands to a node.
+* Creation of a crate called `test-harness` which uses `harness` to write tests for nodes. This
+crate should be capable of starting multiple nodes to test their interaction.
+
+## Phase 2: Networking
+* Definition of the data structures sent as messages over the network.
+* Implementation of a listener in the node which processes these messages.
+* Notifying other nodes when a block has been written.
+* Implementing distributed locking via leader election. 
+
+## Phase 3: Programming Interface
+* Define a package format for user code. This should consist of a WebAssembly binary along with
+a manifest containing metadata about it, such as which principal authored it and which
+permissions the app is requesting. This manifest must be cryptographically signed.
+* Define an API for user code and which exposes the following capabilities:
+  1. Read and write blocks (with permissions checks).
+  2. Send messages to other nodes (which may or may not be in the same tree).
+  3. Lock a file for exclusive access.
+  4. Register a callback for when a file is modified by someone else.
+  5. Make HTTP requests.
+  6. Register callbacks to handle HTTP requests.
+  7. Open arbitrary network sockets.
+(Should this API extend WASI?)
+* Implement a crate which can run WebAssembly code in a sandbox.
+* Define an IPC interface between the sandbox and the node.
+
+## Phase 4: Blockchain
+* Define the data structures needed to construct a blockchain.
+* Implementation the mining algorithm in the node networking module.
+* Implementation of the logic needed to determine the level of redundancy desired for a given block.
+* Implementation an abstraction layer for Erasure Encoding. Define high level functions for creating
+Fragments from a Block.
+* Implementation of the advertising, negotiation and agreement process by which nodes store
+fragments with other nodes.
+* Implementation a Fragment manager in node which handles persisting fragments to the
+filesystem and retrieving them when needed.
+* Registering an endpoint (IP address and port) for a principal.
+* Creation of an index which is created from the block tree and which allows efficient look up of
+node IP addresses and the holders of the fragments created by a given path. This index is updated
+every time a new block is added.
+* When a new block is added, the transaction fees from it are collected and new currency is created
+for each new fragment which is stored. These funds are then transferred to the principal who
+mined the new block.
+
+## Phase 5: Integration
+* A webapp which provides a user interface for nodes. This should be able to:
+  1. Create a new root key.
+  2. Backup and restore a password protected root key to external storage.
+  3. View and approve write cap requests.
+* Creation of a linux distribution to run the node and webapp.
+* A FUSE filesystem daemon for mounting a block tree in a filesystem.
+* An Android app which runs a node.
+
+## Phase 6: Essential Applications
+* App for visualizing and managing a block tree.
+* A contacts app.
+* An object oriented database.
+* A micro-blogging app.
+* A picture/video app.
+* A voice/video chat app.
+
+## Phase 7: Hardware
+* Design and manufacture a router with a hard drive that runs the linux distribution created
+previously.
+* Create a custom Android distribution which runs a node on start up.
+* Design and manufacture a phone which runs this Android distribution.
+
+## Phase 8: The Multiverse
+* Create an app which functions as a server for a metaverse. This app will manage objects in a
+metaverse.
+* Create a client that run under Windows. The client will be responsible for rendering the object
+data it receives from the app and updating objects based on user input. This client must have VR
+and mouse and keyboard support.
+
 ### Outline of paper
 
 *Overview

+ 8 - 1
crates/node/Cargo.lock

@@ -6,7 +6,7 @@ version = 3
 name = "node"
 version = "0.1.0"
 dependencies = [
- "serde",
+ "serde_block_tree",
 ]
 
 [[package]]
@@ -36,6 +36,13 @@ dependencies = [
  "serde_derive",
 ]
 
+[[package]]
+name = "serde_block_tree"
+version = "0.1.0"
+dependencies = [
+ "serde",
+]
+
 [[package]]
 name = "serde_derive"
 version = "1.0.136"

+ 1 - 1
crates/node/Cargo.toml

@@ -7,4 +7,4 @@ edition = "2018"
 # See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
 
 [dependencies]
-serde = { version = "1.0.136", features = ["derive"] }
+serde_block_tree = { path = "../serde_block_tree" }

+ 0 - 2
crates/node/src/main.rs

@@ -1,5 +1,3 @@
-mod serde_blocktree;
-
 use std::collections::HashMap;
 
 /// A Block tagged with its version number.

+ 45 - 0
crates/serde_block_tree/.vscode/launch.json

@@ -0,0 +1,45 @@
+{
+    // Use IntelliSense to learn about possible attributes.
+    // Hover to view descriptions of existing attributes.
+    // For more information, visit: https://go.microsoft.com/fwlink/?linkid=830387
+    "version": "0.2.0",
+    "configurations": [
+        {
+            "type": "lldb",
+            "request": "launch",
+            "name": "Debug executable 'node'",
+            "cargo": {
+                "args": [
+                    "build",
+                    "--bin=node",
+                    "--package=node"
+                ],
+                "filter": {
+                    "name": "node",
+                    "kind": "bin"
+                }
+            },
+            "args": [],
+            "cwd": "${workspaceFolder}"
+        },
+        {
+            "type": "lldb",
+            "request": "launch",
+            "name": "Debug unit tests in executable 'node'",
+            "cargo": {
+                "args": [
+                    "test",
+                    "--no-run",
+                    "--bin=node",
+                    "--package=node"
+                ],
+                "filter": {
+                    "name": "node",
+                    "kind": "bin"
+                }
+            },
+            "args": [],
+            "cwd": "${workspaceFolder}"
+        }
+    ]
+}

+ 8 - 0
crates/serde_block_tree/.vscode/settings.json

@@ -0,0 +1,8 @@
+{
+    "cSpell.words": [
+        "Asym",
+        "blocktree",
+        "newtype",
+        "Xsalsa"
+    ]
+}

+ 25 - 0
crates/serde_block_tree/.vscode/tasks.json

@@ -0,0 +1,25 @@
+{
+    // See https://go.microsoft.com/fwlink/?LinkId=733558
+    // for the documentation about the tasks.json format
+    "version": "2.0.0",
+    "tasks": [
+        {
+            "type": "cargo",
+            "subcommand": "test",
+            "problemMatcher": [
+                "$rustc"
+            ],
+            "group": "test",
+            "label": "test"
+        },
+        {
+            "type": "cargo",
+            "subcommand": "build",
+            "problemMatcher": [
+                "$rustc"
+            ],
+            "group": "build",
+            "label": "cargo build"
+        }
+    ]
+}

+ 65 - 0
crates/serde_block_tree/Cargo.lock

@@ -0,0 +1,65 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = "proc-macro2"
+version = "1.0.36"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c7342d5883fbccae1cc37a2353b09c87c9b0f3afd73f5fb9bba687a1f733b029"
+dependencies = [
+ "unicode-xid",
+]
+
+[[package]]
+name = "quote"
+version = "1.0.17"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "632d02bff7f874a36f33ea8bb416cd484b90cc66c1194b1a1110d067a7013f58"
+dependencies = [
+ "proc-macro2",
+]
+
+[[package]]
+name = "serde"
+version = "1.0.136"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ce31e24b01e1e524df96f1c2fdd054405f8d7376249a5110886fb4b658484789"
+dependencies = [
+ "serde_derive",
+]
+
+[[package]]
+name = "serde_block_tree"
+version = "0.1.0"
+dependencies = [
+ "serde",
+]
+
+[[package]]
+name = "serde_derive"
+version = "1.0.136"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "08597e7152fcd306f41838ed3e37be9eaeed2b61c42e2117266a554fab4662f9"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "syn",
+]
+
+[[package]]
+name = "syn"
+version = "1.0.90"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "704df27628939572cd88d33f171cd6f896f4eaca85252c6e0a72d8d8287ee86f"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "unicode-xid",
+]
+
+[[package]]
+name = "unicode-xid"
+version = "0.2.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "8ccb82d61f80a663efe1f787a51b16b5a51e3314d6ac365b08639f52387b33f3"

+ 10 - 0
crates/serde_block_tree/Cargo.toml

@@ -0,0 +1,10 @@
+[package]
+name = "serde_block_tree"
+version = "0.1.0"
+authors = ["Matthew Carr <mdcarr941@gmail.com>"]
+edition = "2021"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[dependencies]
+serde = { version = "1.0.136", features = ["derive"] }

+ 0 - 0
crates/node/src/serde_blocktree/de.rs → crates/serde_block_tree/src/de.rs


+ 0 - 0
crates/node/src/serde_blocktree/error.rs → crates/serde_block_tree/src/error.rs


+ 2 - 0
crates/node/src/serde_blocktree/mod.rs → crates/serde_block_tree/src/lib.rs

@@ -1,3 +1,5 @@
+/// This crate defines a compact binary serialization format for use in the Block Tree system.
+
 mod error;
 mod ser;
 mod de;

+ 0 - 0
crates/node/src/serde_blocktree/ser.rs → crates/serde_block_tree/src/ser.rs