Securepub uses Userbase ↗︎ users and databases to hold all engagement data.
Here is a high-level summary. See Userbase Architecture↗︎ for in-depth details.
Userbase Users
Each Userbase user may create and manage as many databases as they like subject to practical limits. Every Userbase database is owned by a single user. Users may share their databases with other users. Userbase provides a resharing option to allow the user a database is shared with to share that database with other users as well.
The Userbase getDatabases ↗︎ function allows a user obtain a list of databases they own as well as databases others have shared with them.
Userbase Databases
Each Userbase database holds a primitive collection of json-serialized objects called items ↗︎ limited to 10 KiB bytes or about 5,120 characters each (the character limit is significant because of the way Userbase serializes objects to UTF-16 strings prior to encryption).
Since each item has a key called an itemId, it's easy to imagine Userbase databases as key/value tables but it would be more accurate to think of them as shallow shared folders. This is because Userbase provides no query language or index access methods to retrieve subsets of data.
Databases must be opened with openDatabase ↗︎ before use. Opening a database immediately retrieves all the items in it and although each item has an itemId key there is no API to retrieve only a single item using it. In Userbase opening a database is an all-or-nothing proposition.
Opening a database sets up a Javascript changeHandler callback which Userbase will call when any client changes any item in the database. This is surprisingly powerful and makes it very easy to build a simple lobby, cache or display that updates when there is new data. Unfortunately the callback doesn't tell you which items have changed. It just gives you a list of all of them again so you have to provide your own change detection if you need that (but thankfully items contain some metadata to make this task easier).
Furthermore each call to openDatabase replaces any previous change handler for that database with the new one. Applications relying on this change handler functionality must keep careful track of databases they've opened.
To read and write data of non-trivial size Userbase offers files ↗︎. Each item may have an attached file which can hold 1 or more bytes of data (empty files are not permitted). Clients may upload files to items and read parts back using a file's fileId.
Userbase clients also need to beware that opening a non-existent database in Userbase creates the database so care should be taken to first check if a database exists before attempting to opening it. This is more important than it may seem because Userbase provides no way to drop a database ↗︎ apart from deleting the user that owns it.
There is also no way to rename a database. While Userbase provides a limited form of transaction which ensures atomicity for changes up to 10 items in a single database, it offers no transaction support for operations spanning multiple databases. Userbase applications which need to provide cleanup operations and rollback of partial updates to multiple databases must implement that logic on their own.
Securepub
Userbase does not provide much in the way of row-level security so in order to meet its requirements Securepub partitions engagement information among multiple Userbase users and databases. Securepub relies on Userbase to enforce access controls.
Securepub creates Userbase users when
- an engagement is created
- a member is added
Securepub creates and populates Userbase databases when
- an engagement is created
- a bundle is added or updated
- a topic is added or updated
- a member is added or updated
- a bundle is shared or unshared with a member
- a member visits or reviews a topic
- a member comments on a topic
More details about the users and databases Securepub creates follow.
Create Engagement
Users
Securepub creates a Userbase user for the host when the host creates an engagement. The user is initially given a random username and password. This user will own the host's databases in the engagement.
Databases
Securepub creates the following databases when the host creates an engagement:
-
User (1 db per member)
This is the User database for the host, writable only by the host but shared readable by other members when they are added. The ULID of this database is a prefix for some other databases such as the host's ULID-Role database. The host's User database holds their public profile as well as a topic record for each topic the host creates.
-
ULID-Role (1 db per member)
This is the ULID-Role database for the host, readable and writable only by the host.
The name uses the ULID of the host's User database as a prefix.
It contains precisely one item called the Role record.The ULID-Role database is the starting point to reach other databases in an engagement.
The databaseId of the ULID-Role database is part of the host's invitation link. When the host joins the engagement with their invitation link, Securepub parses the link and tries to open the corresponding ULID-Role database to read the host's Role record. Securepub opens other databases such as User, Members and Bundles following the ids in the Role record.
-
Members (1 db per engagement)
This is the Members database for the engagement, writable only by the host but shared readable by other members when they are added.Each record in the Members database represents a member of the engagement. These records hold
- each member's mnum (member number).
- each member's role.
- each member's Userbase userid.
- each member's User databaseId.
The member's mnum is used as a key internally and in URL fragments. It is a monotonically increasing integer starting at 1. The host always has mnum 1.
-
Links (1 db per engagement)
This is the Links database for the host, readable and writable only by the host.
The host's Links database holds the invitation links for each member.
-
Notes (1 db per member)
This is the Notes database for the host, readable and writable only by the host.
The host's Notes database holds one record for each "member note" recorded about another member.
-
Bundles (1 db per engagement)
This is the Bundles database for the engagement, readable and writable only by the host
It contains one item for each bundle uploaded by the host.
The "file" associated with each item holds the settings for the bundle.Each record in the Bundles database holds a number of attributes:
- the bnum (bundle number) of the bundle.
- the BID (bundle ID) of the bundle.
- the databaseId of the BID-Data database where the bundle's zip data is kept.
- the databaseId of the BID-Entries database holding the bundle's metadata.
- the databaseIds of TID-Topic databases for each guest with bundle access.
- the name and description of the bundle.
- whether or not the bundle is restricted or unrestricted.
- a list of the member numbers of members which may access the bundle.
- bundle statistics including number of folders, number of files and total size.
The bundles's bnum is used as a key internally and in URL fragments. It is a monotonically increasing integer starting at 1.
Add Bundle
Databases
Securepub creates the following databases when the host adds a bundle to the engagement:
-
BID-Data (1 db per bundle)
This is the BID-Data database for the bundle, writable only by the host and readable by any accepted member who the host shares the corresponding bundle with. The name uses the randomly generated BID (bundle ID) from the corresponding record in the Bundles database as a prefix.It contains one item whose associated "file" holds the ZIP data for the bundle.
BID-Data databases for unrestricted bundles are shared directly with members' "guest" user. BID-Data databases for restricted bundles are shared with the meber's "escrow" user until the member accepts their invitation when it is and reshared with the members' "guest" user.
-
BID-Entries (1 db per bundle)
This is the BID-Entries database for the bundle, writable only by the host and readable by any member who the host shares the corresponding bundle with. The name uses the randomly generated BID (bundle ID) from the corresponding record in the Bundles database as a prefix.It contains one item whose associated "file" holds a the metadata for the bundle.
Add Topic
Databases
Securepub creates the following databases when a member creates a topic:
-
TID-Topic (1 db per topic)
This is the TID-Topic database for the member to discuss a topic with other members.
This database is owned by the topic creator and shared readable by other members invited to discuss the topic. It contains one topic data item with the topic name and description and one topic member item for each member identifying the MTID-Activity database holding that member's comments.The MTID (member-topic ID) prefix of the MTID-Activity database is generated randomly by the topic creator when they invite a member to comment on a topic. Comments from that member are expected to be found in the corresponding database owned by the member.
Each topic has a topic number which is a monotonically increasing integer starting at 1 for each topic creator. I.e. the first topic created by each member will have topic number 1.
Each topic's tkey (topic key) is constructed from the mnum of the topic's creator and the digits of the topic number transliterated as follows
0 1 2 3 4 5 6 7 8 9 │ │ │ │ │ │ │ │ │ │ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ ▼ Z A B C D E F G H J
For example the tkey "3B" indicates the second topic (B) created by member 3.
The tkey is used as a topic key internally and in URL fragments.
-
TID-Updated
(1 db per topic)
This is the TID-Updated database to notify clients when a member updates a topic.
This database is owned by the topic creator and shared writable by other members invited to discuss the topic. It contains one topic member item for each member who has created their MTID-Activity database.
Add Member
Users
Securepub creates a new Userbase user when the host adds a member. The user is initially given a random username and password. The documentation refers to this user as the "guest" user. This user will own the members's databases in the engagement.
The host records the invitation link for the new member in the Links database.
Securepub also creates another temporary Userbase user for internal use when the host adds a member. The user is also given a random username and password. The credentials are stored in the guest's ULID-Bundles database.
The documentation refers to this user as the "escrow" user . This user serves as an escrow for bundle databases shared with members who have not yet accepted their invitation.
This "escrow" user owns no databases but is granted the special resharing access by the host for restricted bundle BID-Data databases.
When an invited member accepts their invitation and becomes an accepted member, Securepub uses the "escrow" user credentials to reshare restricted bundle databases with the member's regular "guest" user.
This extra sharing step is only necessary for restricted bundles. Unrestricted bundle databases are shared directly with the "guest" user.
Securepub removes the "escrow" user and credentials from the ULID-Bundles database after an invited member accepts their invitation since at that point they are no longer needed. Databases of bundles shared with accepted members are directly shared with the member's regular "guest" user.
Databases
Securepub creates the following databases when the host adds a member to the engagement:
-
User (1 db per member)
This is the User database for the member, writable only by the member but shared readable with the host and other members when they are added. The ULID of this database is a prefix for some other databases such as the member's Role database and their ULID-Bundles databases. The User database holds the members' public profile as well as a topic record for each topic created by the member.
-
ULID-Role
(1 db per member)
This is the ULID-Role database for the member, owned and writable only by the host but readable by the member. The name uses the ULID of the host's User database as a prefix.
It serves the same function as the Host's ULID-Role database described above.
-
Notes (1 db per member)
This is the Notes database for the member readable and writable only by the member.
The member's Notes database holds one record for each "member note" recorded about another member.
-
ULID-Bundles
(1 db per guest)
This is the ULID-Bundles database for the member, owned and writable by the host but readable by the member. It contains one item for each bundle shared with the member with the same attributes as the Host's Bundles database.
Share Bundle
Databases
Securepub creates the following databases when the host shares a bundle with a member:
-
TID-Topic
(1 db per topic)
This is the TID-Topic database for the member to privately discuss the bundle with the host.
This database is owned by the host and shared readable by the member.
It contains precisely one topic data item with the topic name and description and two topic member items identifying the MTID-Activity databases for the host and member.
Visit Topic
Databases
Securepub creates the following databases the first time a member visits a topic:
-
MTID-Activity
(1 db per topic visited by a member)
This is the MTID-Activity database holding comments made by a member on a given topic.
It is owned by the member and shared readable with the other members in the topic.
It contains one item for each comment made by the member. It also contains items recording the number of times the member visited or reviewed the topic on a given date.
Database Diagram
The Securepub databases are shown in the diagram below and explained in the following sections.
AppId
+
RoleID
╵
╵ not shared shared with all not shared not shared (only host)
╵ ╭───────────╮ ╭───────────────╮ ╭───────────╮ ╭───────────╮
└╶▶│ ULID Role │╶╶╶▶│ Members │ │ Notes │ │ Links │
├───────────┤ ├───────────────┤ ├───────────┤ ├───────────┤
│ * role │ │ * nextmember │ │ * note │ │ * link │
╰──────── H ╯ │ * member │ ╰──────── U ╯ ╰──────── H ╯
╵ ╰──────────── H ╯
╵
╵
╵ shared with all
╵ ╭───────────────╮
└╶╶╶╶╶╶╶╶╶╶▶│ User │
╵ ├───────────────┤
╵ │ * nexttopic │
╵ │ * verifymsg │
╵ │ * profile │
╵ │ * topic │
╵ ╰──────────── U ╯
╵ ╵ topic members only
╵ ╵ ╭───────────────╮
╵ └╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶▶│ TID Updated │
╵ ╵ ├───────────────┤
╵ ╵ │ * topicmember │
╵ ╵ ╰──────────── M ╯
╵ ╵
╵ ╵ topic members only
╵ ╵ ╭───────────────╮
╵ └╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶▶│ TID Topic │
╵ ├───────────────┤
╵ ┌╶╶╶╶▶│ * topicdata │
╵ ╵ │ * topicmember │
╵ ╵ ╰──────────── M ╯
╵ shared with user ╵ ╵ topic members only
╵ ╭───────────────╮ ╵ ╵ ╭───────────────╮
└╶╶╶╶╶╶╶╶╶╶▶│ ULID Bundles │ ╵ └╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶▶│ MTID Activity │
╵ ├───────────────┤ ╵ ├───────────────┤
╵ │ * bundle │╶╶╶┘ │ * visit │
╵ ╰──────────── H ╯ │ * review │
╵ │ * comment │
╵ ╰──────────── M ╯
╵
╵ not shared
╵ ╭───────────────╮
└╶╶╶╶╶╶╶╶╶╶▶│ Bundles │
├───────────────┤ bundle userbase file
│ * nextbundle │ ┌───────────────┐
│ * bundle - - - - - - - - - - - - - - - - - - │ settings file │
╰──────────── H ╯ └───────────────┘
╵
╵ bundle members only
╵ ╭──────────────╮
└╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶▶│ BID Data │ biddata userbase file
╵ ├──────────────┤ ┌──────────┐
╵ │ * biddata - - - - │ zip file │
╵ ╰─────────── H ╯ └──────────┘
╵
╵
╵ bundle members only
╵ ╭──────────────╮
└╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶▶│ BID Entries │ bidentries userbase file
├──────────────┤ ┌────────┬──────┬─────┐
│ * bidentries - - - │ header │ page │ ... │
╰─────────── H ╯ └────────┴──────┴─────┘
╭──────────────────────────────────────────────────────────────────────────────────────────────╮
│ legend │
│ sharing │
│ ╭─────────────────╮ │
│ │ parent database │ a record may have an associated file │
│ ├─────────────────┤ ┌────────────────┐ │
│ │ * record type - - - - - - - │ userbase file │ │
│ ╰────────── owner ╯ └────────────────┘ │
│ ╵ │
│ ╵ a database may have child databases whose │
│ ╵ records cannot exist without corresponding │
│ ╵ records in their parent database │
│ ╵ ╭────────────────╮ │
│ └╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶╶▶│ child database │ │
│ ┌╶╶╶▶│ │ │
│ ╵ ╰────────────────╯ │
│ ╵ │
│ ╭─────────────────────╮ ╵ each child database has one parent and │
│ │ associated database │╶╶┘ zero or more other associated databases │
│ ╰────────────── owner ╯ which refer to it │
│ │
│ e.g. each TID-Topic has a parent User and │
│ some TID-Topics also have associated bundles │
│ where │
│ │
│ sharing = not shared database = Notes record type = note │
│ | shared with all | Links | link │
│ | shared with user | ULID-Role | role │
│ | topic members only | Members | nextmember │
│ | bundle members only | User | member │
│ | TID-Topic | verifymsg │
│ | TID-Updated | profile │
│ | TID-Topic | nexttopic │
│ | MTID-Activity | topic │
│ owner = H ╶╶╶ host | Bundles | topicdata │
│ | U ╶╶╶ user | BID-Data | topicmember │
│ | M ╶╶╶ topic member | BID-Entries | nextbundle │
│ | ULID-Bundles | bundle │
│ | biddata │
│ | bidentries │
│ userbase file = settings file | visit │
│ | zip file | review │
│ | entries file | comment │
│ │
╰──────────────────────────────────────────────────────────────────────────────────────────────╯