CAN

The can (Content Addressable Notebook) project is a note-taking system inspired by the unison programming language / codebase manager.

• What would be included in a first iteration of CAN?
• Links in CAN

Backlinks

• Links in CAN
  • Hyperlinks between notes will be a core part of CAN.
• The CAN manager init command
  • The init command will create a CAN notebook directory in the current working directory (the CAN notebook is stored in the .can directory). The command will only be available if the directory doesn't exist at start-up (CAN should check if a notebook directory exists at start-up).
• CANs link command should enumerate links for the named note
  • When called with a valid note name as an argument, the CAN link command should enumerate links for the named note.
• CAN notes are written in a subset of markdown
  • Although CAN is able to parse and store full markdown (CAN will use remark to process input), to speed up development and avoid thinking through potential pitfalls of full markdown support until later CAN will use a subset of markdown for now.
• CAN manager links should be rendered as blue text
  • The CAN manager displays rendered notes to the user. Rather than displaying raw markdown syntax for hyperlinks, CAN should indicate that a section of text is a link through colour.
• CAN is a REPL-like environment
  • The intent is that CAN's notebook manager would be open while the user works on their notes, alongside any text editor opened on scratch.z. The manager provides the ability to view, search, explore, export, create, edit and organise a CAN notebook through a series of commands.
• CAN will use remark to process markdown input
  • Remark provides an interface for producing, modifying and extending a markdown AST using itself and unist tooling. This should allow CAN to get off the ground and stay extensible over development.
• A CAN note is stored using its AST
  • Rather than storing plain-text, a note in CAN will be stored using an abstract, semantic representation (an Abstract Syntax Tree). This allows both input and output to be extended easily to use different formats and also allows robust & simple internal tooling.
• The CAN list command should show the names of your notes
  • The CAN list command should show the name of each note currently in the notebook.
• CAN will use multihash to encode digests
  • The CAN notebook will address notes using multihash encoded digests. This will allow future support for fallbacks in the case of a problem - as CAN will depend on openssl for hashing algorithms
• Each CAN address is a directory
  • Inside CAN's notebook there should exist a directory for every address present. The directory will be named the hash of the note which lives at this address (CAN will use SHA3-512 digests). Inside the address there'll be an AST file with the content of the note itself and a metadata file alongside the AST.
• CAN will use SHA3 512-bit digests
  • The CAN notebook addresses will be SHA3-512 multihashes (CAN will use multihash to encode digests)
• When the SCRATCHFILE_PARSED event is fired CAN should hash the AST
  • CAN should fire a SCRATCHFILE_PARSED event. When this happens, CAN should make a digest of the AST. CAN will use SHA3-512 digests and will use multihash to encode digests. For now no check needs to be performed around which hashing algorithm is used, as there's only one implemented.
• Content Addressable Note-taking system
  • This is a project I'll be working on. Called CAN for want of a better name.
• CAN should contain a customised instance of the remark markdown serialiser
  • CAN uses remark to render markdown both for display and for export.
• CAN should fire a SCRATCHFILE_ERROR event if unsupported syntax is found
  • CAN should parse the scratchfile when it is updated. If CAN hits an error in the scratchfile or finds unsupported syntax (CAN uses a subset of markdown), it should fire a SCRATCHFILE_ERROR event with the error passed as an argument. To allow other parts of the software to respond.
• Each CAN address will contain a metadata file
  • In every address directory inside a CAN notebook will exist a metadata file.
• CAN's interface will be built using vorpal
  • The interface for CAN's notebook manager repl-like interface will be built using vorpal. This gives me the interface I want with little nice bits like tab-completion for free.
• CAN will not change the filesystem without a user command
  • In order to help ensure data security, CAN will not create or edit a single file or directory without a command from the user. It also won't access the network. The worst it will do is watch the scratchfile when it starts up (The CAN scratchfile is scratch.z).
• Each CAN address will contain an AST file
  • CAN's notebook has a directory for each address, and each note has its own address. Within this address exists a file with the name ast, containing the AST of the note stored here, represented in JSON.
• CAN should parse the scratchfile when it is updated
  • In order to be able to store notes and use content-addressable storage, CAN needs to have access to the note's content. Because of this, CAN should parse the scratchfile whenever it is updated (CAN should fire a SCRATCHFILE_UPDATED event when the scratchfile is updated). Once the AST has been constructed, CAN should ignore 'beneath the fold'.
• CAN should inform the user of changes made after a command runs
  • CAN should build trust that it only makes sensible changes that are within its remit. It should also let the user build an accurate mental model of what each command does. For similar reasons, CAN will not change the file system without a user command.
• When a scratchfile is created CAN should start to watch it
  • To manage this, an event handler pattern will be used. Commands will be given access to emitters for specific events, and the entry file for CAN will set up handlers for the events.
• CAN note names are letters, numbers, and dashes
  • CAN's note names don't have spaces. They contain only alphanumeric characters and dashes.
• CAN note-existence handler should emit NOTE_EXISTENCE
  • So other systems can respond when a note is identified in CAN, the note-existence handler will fire a NOTE_EXISTENCE event with the following argument
• CAN should fire a SCRATCHFILE_PARSED event when the scratchfile is parsed
  • CAN should parse the scratchfile once it has been updated. Once CAN has successfully parsed the scratchfile, the SCRATCHFILE_PARSED event should be fired, with the AST included as an argument. To allow other parts of the software to respond.
• CAN should store and address notes semantically
  • CAN's focus is on the meaningful content of notes rather than presentation. This will allow CAN to accept different input formats and output in multiple formats too. CAN will still be able to transform and work with the parts of the note that matter.
• The add command should tame a name as an argument
  • CAN's add command takes the name of the new note as an argument. CAN note names are letters, numbers and dashes
• CAN will use marked to process markdown input
  • Marked provides CAN with the flexibility to use its AST in early development while exposing a stand-alone lexer if the parser needs replacing in the future (marked provides a lexer, parser, and renderer for markdown).
• The CAN store's current scratchfile address field holds the address of everything above-the-fold in the scratchfile
  • The CAN state-store currentscratchfileaddress field holds the last-updated address of everything above the fold in the scratchfile
• CAN should fully stringify the AST before creating a digest
  • When addressing a note, CAN should make sure that the AST has been stringified in full to avoid lost information.
• CAN's inform-add handler should listen to NOTE_EXISTENCE
  • CAN's inform-add handler should inform the user of the existence of the add command when the NOTE_EXISTENCE event is fired and the address field matches the current scratchfile address.
• CAN's add command should fire a COMMAND_ADD event
  • CAN's add command should trigger a COMMAND_ADD event, with the name provided as an argument.
• The CAN store's current scratchfile new field holds if the current scratchfile exists in the notebook
  • The CAN's current scratchfile new holds if the current scratchfile is currently a note within the notebook.
• CAN should fire a SCRATCHFILE_FOUND event when an existing scratchfile is found
  • CAN may look for existing scratchfiles at different stages in the future. When one is found, a SCRATCHFILE_FOUND event should be fired, with the path of the scratchfile as an argument.
• CAN's watch-scratchfile handler should listen to the SCRATCHFILE_FOUND event
  • So all scratchfiles found by CAN are watched for changes, the watch scratchfile handler should listen to the SCRATCHFILE_FOUND event.
• CAN should fire a SCRATCHFILE_UPDATED event when the scratchfile is updated
  • When a scratchfile is created CAN should start to watch it. When CAN detects a change in the scratchfile, a SCRATCHFILE_UPDATED event should be emitted.
• CAN should ignore 'beneath the fold'
  • CAN's scratchfiles may have a marker (---) called the fold. Anything beneath this point should be ignored by can. This gives the user space to keep short-lived working drafts while they explore their notes.
• When a SCRATCHFILE_ERROR is emitted, CAN should inform the user
  • CAN should fire a SCRATCHFILE_ERROR event if unsupported syntax is found. When this happens, the user should be informed. For now, CAN should simply print the error with some context 'parsing scratchfile failed'.
• CAN should have a single state store
  • So CAN's handlers can access application state when needed, CAN should have a state store. The state store sdould listen to events to update its state when needed, and should allow read-only queries of its state.
• CAN's create-note handler should fire NOTE-CREATED-ERROR on failure
  • CAN's create-note handler will attempt to create a new note at an address in the notebook. If this goes wrong for any reason, the handler should fire a NOTE-CREATED-ERROR event to allow other handlers to listen and respond.
• CAN's create-note handler should fire NOTE_CREATED
  • The create-note handler is intended to create a new note at the correct address within the CAN notebook.
• CANs note-existence handler should listen for SCRATCHFILE_ADDRESSED
  • So CAN can learn if the scratchfile contains a new note. The note-existence handler should listen to the SCRATCHFILE_ADDRESSED event. Once recieved, the handler should check if a note at this address exists in the notebook. CAN's note-existence handler should fire NOTE_EXISTENCE when it determines if a note exists.
• CAN's store should listen to NOTE_EXISTENCE
  • When CAN's state store detects a NOTE_EXISTENCE event it should update the currentscratchfilenew field if the event address matches address of currentscratchfileaddress. The CAN store's currentscratchfilenew field holds if the current scratchfile exists in the notebook.
• CAN's store should listen to SCRATCHFILE_ADDRESSED
  • When CAN's state store detects a SCRATCHFILE_ADDRESSED event, it should update the currentscratchfileaddress field. The CAN store's currentscratchfileaddress field holds the address of everything above-the-fold in the scratchfile
• CAN's create-note handler should listen for COMMAND_ADD
  • CAN's create-note handler is intended to add a new note at the note's address using the current state of the scratchfile to determine which note is added.
• When a CAN scratchfile is addressed, check to see if the note is new
  • This helps CAN decide what to tell the user; and how commands should respond based on context. CAN should have a single state store which can be queried about the latest state of the scratchfile. Commands won't be able to respond to CAN events so state becomes necessary at this point.
• CAN's scratch command should fire the COMMAND_SCRATCH event
  • When a CAN user sends a scratch command, the COMMAND_SCRATCH event should be fired to allow handlers to respond. This should include whether the scratchfile exists as a boolean argument.
• CAN's commands should live in the commands directory
  • CAN's commands should all exist within the commands directory to allow a clear place to place new commands.
• CAN's link command should fire COMMAND_LINK
  • CAN's link command should fire a COMMAND_LINK. The event should fire with an object as an argument:
• The CAN list command should take an argument to filter note names
  • CAN's list command should take an optional string argument. When present, this argument should fuzzy-search the names of notes, with the command listing any matches.
• CAN may not recognise changes to your scratchfile if your editor has safewrite
  • I've found that CAN may not recognise changes to a scratchfile. Turns out it is an issue with safewrite - vim safewrite stops file watching from working correctly
• CAN's scratch command should create a scratchfile if none exists
  • The scratch command within CAN is intended to prepare the scratchfile for writing a fresh note. If a scratchfile doesn't exist, it should create it and inform the user (CAN should inform the user of changes after a scratchfile is run).
• CAN's scratch command should push existing scratchfile content below the fold
  • Running CAN's scratch command while a scratchfile exists should push all existing content beneath a new fold. This is done by adding --- to the top of the scratchfile.
• The CAN search command should fuzzy search all note content
  • CAN's search command should search the text content of all notes with the string provided. The results should list each note that has matching text. Each note listed should have a title first, followed by every line which contains a match within that note. The sections which match directly should be highlighted.
• CAN should begin watching an existing scratchfile on startup
  • To avoid the user having to remove and recreate the scratchfile for each session of CAN, CAN should check for and watch an existing scratchfile at startup time.
• Can should inform the user of the add command when a new note is in the scratchfile
  • CAN's inform-add handler should listen to NOTE_EXISTENCE, informing the user of the add command. The add command should exist at this point and should fire a COMMAND_ADD event with the user-passed name.
• CAN should keep a name to address index
  • CAN's notebook should keep an up-to-date index of note names to addresses, to allow for situations where CAN needs to map a name to an address.
• CAN's add command should save the current scratchfile as a note.
  • CAN's add command is intended to add new notes to the notebook. When a new note has been detected and the add command is run, this new note should be created at the correct address. Each address in a CAN notebook is a directory. Each CAN address contains the AST of the note at this address and a metadata file containing names.
• CAN's init command should inform the user when successful
  • CAN should always respond with the changes that have been made when a command is successful (CAN should inform the user of changes made after a command runs)
• CAN's notebook address content
  • Every address in a CAN notebook is a directory.
• Which hashing algorithm should the CAN manager use?
  • CAN stores notes in the notebook at addresses based on their content, rather than by user-provided names. These addresses will be generated using cryptographic hashing algorithms.