diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md new file mode 100644 index 0000000..9ef0652 --- /dev/null +++ b/ARCHITECTURE.md @@ -0,0 +1,149 @@ +# Base. + +### `PCH` + +Brings in all the external and standard headers.
+Apart from c++std stuff we have: +- `mruby` and almost all it's subheaders. +- `pcre2` (For regex.) +- `grapheme.h` - The suckless libgrapheme (to find utf8 boundaries) +- `unicode_width.h` - To find terminal width of unicode char's. + +*Maybe try removing some of them ive probably stopped using.* + +### `definitions.h` + +- Defines the `bed` namespace +- Forward declaration of `BEd` type needed by all. +- `fatal_error` definition, a msg, code (quits application on being thrown) +- `ed_error`. just a message (stops the current cycle of the ed repl.) + +### `bed.h` + +**As this is based heavily on the internals, and so ill look into it at the end.** + +# `Internal` + +## `Vase` + +- This module doesn't depend on any other. +- It's namespace is `bed::internal::vase` +- Vase is the module to do with the actual storage and handling of a disk backed immutable and reference counted avl piece tree. +- It is a O(log n) per edit text datatype. + +### `constants.h` + +Defines `PETAL_SIZE_MAX` as `32 * 1024`. +- `PETAL_SIZE_MAX` limits teh size of a petal which makes multiple O(n) in petal searches O(1) as it is now bounded. + +### `storage/` + +A storage is something that stores the actual text, the text is actually stored in storages which are referenced by the trees. + +#### `storage.h` + +Defines the virtual `Storage` class. +It has the methods: +- `const char *read(uint64_t pos)`: reads text from position `pos` and returns a pointer to it. +- `uint64_t length()`: returns the length of the storage. +- `void retain() / release()`: Methods to help with refcounted storages. + +#### `original.h` + +Defines the `Original` type of storage. +This is a refcounted storage. + +It has the fields: +- `atomic_uint64_t refs{0}`: refcount +- `const char *buf`: pointer to the actual text. +- `uint64_t len` +- `int fd = -1`: the file descriptor of teh undelying disk storage. + +It adds the method `initialize` over the base class. + +`src/vase/storage/original.cc`. + +The constructor uses `mkstemp` to make a temp file, then unlinks it to hide it from the fs but keeps the fd alive. + +`initialize` then `mmap`s the fd into a non modifyable pointer managed by the kernel then closes the fd itself. +- This is what gives us a pointer to the file contents that the kernel can load and evict from memory as needed, and therefore if we have say a 500MB file loaded, but the cursor is arount 200MB and the user is only editing a couple lines around that, the rest of the file will not be using your memory, it might be loaded but if needed the kernel can evict it. + +The call site fills the fd before calling initialze, it cannot be modified after that. + +#### `append.h` + +Defines the `Append` type of storage. +This is not actually refcounted, it is to be owned by the application just once and reused all throughout. +It contains the actively typed stuff. + +It has the fields: +- `const char *buf`: pointer to the actual text. +- `uint64_t allocated_capacity`: the amount of bytees that can be written to it without needing to increase the size. +- `uint64_t current_size`: Stores the size of actual text stored in it. (the cursor) +- `int fd = -1`: the file descriptor of the underlying disk storage. + +It has the methods `append` (with 2 overloads) and private `grow` over the base class. +`retain` and `release` do nothing. + +`src/vase/storage/append.cc`. + +The constructor uses `mkstemp` an `unlink` similar to the original storage. +It then `ftruncate`s the file to `2^30` bytes or 1GiB this makes writing 1gb to the file possible but does'nt neccasarily use up 1gb on the disk. +But when `mmap`ing we set `PROT_READ | PROT_WRITE` and `MAP_SHARED` to be able to write edits to disk. + +`grow` works by doubling the capacity `ftruncat`ing the fd to the new capacity and then on `linux` it uses `mremap` to remap to the new file size, but otherwise we `munmap` and `mmap` into the new size. + +The append methods append text into the buffer (either a single char or char * + len), they may grow the storage size. (the char*+len version uses memcpy). + +### `shard.h` + +Defines the actual peice tree. + +The tree is a polymorhic set of classes. + +The base class `Shard` has: +- `enum Kind : uint8_t kind`: Branch or Petal. +- `uint16_t height`: the tree height (for avl balancing.) +- `atomic_uint32_t refs`: the refcount. +- `uint64_t length`: the length in bytes of this (sub)tree. +- `uint64_t lines`: the number of `\n` in this subtree. +- It starts refcount at 1. + +The `Branch` class adds: +- `Shard *left/right` the subtrees. +- its constructor takes l, r and sets length/lines `(l->length + r->length)` +- and height `1 + std::max(l->height, r->height)` + +The `Petal` (leaf) class adds: +- `Storage *` a pointer to the storage used. +- `uint64_t pos` the position in the storage its text starts at. +- it takes `(uint64_t length, uint64_t lines, Storage *source, uint64_t pos)` +- it retains `source`. (usefull for refcounted sotrages.) + +A `nullptr` `Shard*` is an empty peice of text (i.e. valid). + +`src/internal/vase/shard.cc`: defines a set of static function on the Shard class. (All the applications of it.) + +the operations methods are: +- `retain/release` retain/releases them, if freeing a leaf it also releases the storage and for branches it releases both subtrees. nullptr's are ignored. +- `balance/rotate_(right/left) and height/balance factor` are for doing simple avl balancing operations. +- `concat` borrows 2 pointers to `Shard` and returns an owned Shard*. +- `split` borrows a `Shard` and returns a `pair` of 2 owned shards. +- `merge_leaves` works similar to concat but it tries to merge the middle bit if they have their physical representation in order. for example if i have a shard `hell` and i type the letter `o` and because i'd been typing in order the Append storage will have text `hello` we can merge them into a single petal. +- `append` similar to concat, but uses merge_leaves, useful for small appends like when typing. +- `build` takes a pointer to an array of `Shard*` and start and end then makes a single avl balanced `Shard*` out of it it is generally faster than calling concat on each peice when we have many. + +Then for building a Shard* in the first place we have: +- `from_command(const char *cmd)` runs the \0 terminated command string and reads its STDOUT and returns an owned Shard*. +- `from_file(const std::filesystem::path &path)` reads the file at path. +- `from_string(const char *data, uint64_t len)` loads the string given into a new storage, (normally inserting text can work by inserting into the append storage but this function loads it into a `OriginalStorage` object.). +These methods all clip the final newline if it exists. +And they also create a new `OriginalStorage` object. + +### `vase.h` + +### `iterators/` + +#### `line.h` + +#### `petal.h` diff --git a/README.md b/README.md index 6ecdae5..e854d69 100644 --- a/README.md +++ b/README.md @@ -56,9 +56,7 @@ It should support: - Error handling. - And more. -Not done yet. - -### TODO immediately: +### TODO: - Make "g" command work. - properly handle escapes for %q ' etc in ruby parser (rn everything is escapable.)