Documenta2

Next iteration of the Documenta collaborative project documentation system

Current version

Documenta was born out of our frustration with current solutions for collaborative documentation of lab projects. There are plenty of Wiki and CMS apps out there but they all lack the simplicity and immediacy needed for documenting projects on the spot.

The main idea behind documenta is that since most people know how to write and organize text files on their personal computers, we could leverage this knowledge to build an intuitive web publishing system.

The current version of Documenta has been in use for about a year now. It has been cobbled up from various existing applications:

The cool thing about this version is that it took very little effort to put together, allowing us to quickly evaluate its usefulness and understand where it can be improved.

Using file trees to organize content for publishing has proven to be a very useful metaphor, allowing for easy introduction to new users while providing a very efficient workflow for more experienced users. Having the ability to write and organize files on the users' computer and then publish changes automatically via WebDAV or ownCloud's file synhronization API has also proven to be very productive.

Using Markdown files for content has been a mixed success. While editing text files is something that all our users do easily, we found that writing Markdown is not everyone's cup of tea. The addition of the online editor with button shortcuts and content preview helped a lot in teaching the syntax to new users.

We also found some some issues with the current setup:

  • no history (it is hard to know what is new and what has been changed)
  • no locking (editing shared documents is a mess)
  • no read access control (all documents are public)
  • no way to give feedback on documents (i.e comments)
  • no integration with lab services (e.g. badges)

Goals for version 2

There are two major goals for the new version of the documenta platform:

  • Improve the user experience

    • make mobile data capture and note taking a no-frills experience
    • support the creation of development stories from project data:
      • modular content blocks
      • presentation templates
      • project timeline

  • Create solid document storage platform

    • API-first design
    • versioned file trees with embedded metadata
    • easy to integrate with external services
    • easy to extend (e.g. to support comments)
    • built-in support for real-time collaboration

System design

Documenta2 system architecture

At the heart of the system is the Repository API which provides a REST interface to the documentation data under a familiar file tree data model.

Documenta clients use the Repository API as backend to read and store documentation data.

Repository API

  • accessible via standard HTTP verbs
  • each path has an associated ACL

Retrieving data

  • GET <path> retrieves the rendered representation of the path:
    • GET /u/pangelo/ retrieves the HTML rendering of /u/pangelo/index.md
    • GET /u/pangelo/notes retrieves the HTML rendering of /u/pangelo/notes.md
  • GET <path>.ext retrieves a raw file:
    • GET /u/pangelo/notes.md retrieves the Markdown source file
    • GET /u/pangelo/title.jpg retrieves an image file
  • GET <path>?q=<op> allows retrieving extra information about a file
    • GET /u/pangelo/notes.md?q≃meta retrieves metadata about the file
    • GET /u/pangelo/notes.md?q=hist retrieves file editing history
    • GET /u/pangelo/notes.md?q=edit retrieves the file edit status

Writing data

  • POST <path> creates a new entity in the file tree
    • if <path> is a directory then it is created and an index.md file is placed within
    • if <path> is a file then it is uploaded

Updating data

  • PUT <path> can be used to update existing entities in the file tree
  • PUT <path>?m=<text> can be used to add a revision history message

Deleting data

  • DELETE <path> can be used to delete existing entities in the file tree
  • DELETE <path>?m=<text> can be used to add a revision history message

Documenta2 name space

The repository is viewed as a large file system where:

  • \ is the root of the repository
  • \u\<username> is the repository space for a single user
  • \g\<groupname> is the repository space for a group of users
  • \u\<username>\<projectname> is the repository space for a single project
  • \g\<groupname>\<projectname> is the repository space for a group of users

Open research topics

  • Version files independently (like CVS/WebDAV) or in changesets?
  • Comments
  • Realtime collaborative editing

Project repo

http://github.com/altlab/documenta2/

TODO

  • merge research notes from code/active/labs/documenta2/{ideas.txt, prep/doc/design.txt} into documenta/projects/documenta2

References and Related projects