What is the front-end storage besides localStorage?

Posted Jun 16, 202012 min read

This article introduces some of the most interesting, fun, and useful open source libraries related to front-end data storage. In addition, it will let you master the characteristics of various Web storage solutions, and quickly understand.

In the front-end data storage method, have you used other storage methods besides cookies, localStorage and sessionStorage? In fact, in addition to the three storage methods mentioned above, the current mainstream browsers also support Web SQL and IndexedDB.

At present, the mainstream browsers on the market are Chrome, Safari, Firefox, Opera, UC Browser, and Internet Explorer. Among them, as of May 2020, Chrome's market share is 63.93%, far exceeding the second place Safari Browse(18.19%).

stat-counter-browser-ww-monthly.png

(Photo source: https://gs.statcounter.com/ )

Here we take the Chrome browser with the largest market share as an example to understand all the storage solutions it supports:

web-browser-storage.jpg

(Open Chrome Developer Tools and switch to the Application field)

Before introducing some of the more popular open source front-end storage solutions, A Baoge first shared some interesting and fun open source libraries related to storage.

1. Interesting and fun open source library

1.1 Sharedb

Realtime database backend based on Operational Transformation(OT).

https://github.com/share/sharedb

ShareDB is a real-time database backend based on JSON document operation transformation(OT). It is the real-time backend of the DerbyJS Web application framework.

Example 1:Real-time data synchronization

simple-app-for-realtime-sync.gif

Example 2:A leaderboard application showing real-time queries

leaderboard-for-live-queries.gif

1.2 ImmortalDB

? A relentless key-value store for the browser.

https://github.com/gruns/Immo...

ImmortalDB is the best way to store persistent key-value data in the browser. The data saved to ImmortalDB is redundantly stored in Cookies, IndexedDB and localStorage, and if any of the data is deleted or damaged, they will continue to repair themselves.

For example, clearing cookies is a common user operation, even for non-technical users. Under storage pressure, the browser can delete IndexedDB, localStorage, or sessionStorage at will without warning.

Example

import {ImmortalDB} from'immortal-db'

await ImmortalDB.set('name','semlinker'); //Set
await ImmortalDB.get('name', default='lolo'); //Get
await ImmortalDB.remove('name'); //Remove

1.3 web-storage-cache

Expanded localStorage and sessionStorage, added timeout and serialization methods.

https://github.com/wuchangmin...

WebStorageCache extended HTML5 localStorage and sessionStorage, added timeout, serialization method. You can directly store JSON objects, and at the same time you can easily set the timeout.

Optimization:WebStorageCache automatically clears the expired data accessed, avoiding the accumulation of expired data. In addition, it also provides a method to clear all expired data:wsCache.deleteAllExpires();

Example

var wsCache = new WebStorageCache();

//Cache string'wqteam' to'username', timeout time 100 seconds
wsCache.set('username','wqteam', {exp:100});

//Time-out deadline, Date type can be used
var nextYear = new Date();
nextYear.setFullYear(nextYear.getFullYear() + 1);
wsCache.set('username','wqteam', {exp:nextYear});

//Get the value of'username' in the cache
wsCache.get('username');

//Cache simple js objects. The serialization method is JSON.stringify by default.
//You can configure serializer.serialize when initializing wsCache
wsCache.set('user', {name:'Wu', organization:'wqteam'});

1.4 lz-string

LZ-based compression algorithm for JavaScript.

https://github.com/pieroxy/lz...

lz-string is designed to meet the needs of storing large amounts of data in localStorage(especially on mobile devices). localStorage is usually limited to 5MB ~ 10MB, you can compress the data to store more data.

Example

var string = "Hello, my name is semlinker";
console.log("Size of sample is:"+ string.length);

var compressed = LZString.compress(string);
console.log("Size of compressed sample is:"+ compressed.length);

string = LZString.decompress(compressed);
console.log("Sample is:"+ string);

The following figure is the result of using the official online example string compression test:

lz-string-demo.jpg

(Source: https://pieroxy.net/blog/page... )

Next we start to introduce some mainstream databases.

Second, the mainstream database

2.1 localForage

? Offline storage, improved. Wraps IndexedDB, WebSQL, or localStorage using a simple but powerful API.

https://github.com/localForag...

localForage is a fast and simple JavaScript repository. It uses asynchronous storage(IndexedDB or WebSQL) by using a simple API similar to localStorage to further improve the offline experience of your web application.

For browsers that do not support IndexedDB or WebSQL, localForage uses localStorage for data storage. In addition, localForage also supports storing all native JS objects that can be serialized into JSON, as well as ArrayBuffers, Blob, and TypedArrays.

LocalForage mainly supports platforms:

  • IE 10(IE 8+ uses localStorage)
  • Opera 15(Opera 10.5+ uses localStorage)
  • Firefox 18
  • Safari 3.1(including Mobile Safari)
  • Chrome 23, Chrome for Android 32
  • Phonegap/Apache Cordova 1.2.0

2.2 PouchDB

?-PouchDB is a pocket-sized database.

https://github.com/pouchdb/po...

PouchDB is an in-browser database that allows applications to save data locally so that users can enjoy all the features of the application even when offline. In addition, the data is synchronized between the clients, so users can stay up to date anytime, anywhere.

PouchDB also runs in Node.js and can be used as a direct interface to CouchDB compatible servers. The API works the same in every environment, so you can spend less time worrying about browser differences, and spend more time writing clean, consistent code.

PouchDB supports all modern browsers:

  • Firefox 29+(Including Firefox OS and Firefox for Android)
  • Chrome 30+
  • Safari 5+
  • Internet Explorer 10+
  • Opera 21+
  • Android 4.0+
  • iOS 7.1+
  • Windows Phone 8+

PouchDB uses IndexedDB behind the scenes, if the current environment does not support IndexedDB, then fall back to Web SQL.

2.3 Rxdb

? ? ? A realtime Database for JavaScript Applications.

https://github.com/pubkey/rxdb

RxDB(short for Reactive Database) is a NoSQL database used for JavaScript applications such as websites, hybrid applications, Electron Apps, Progressive Web Apps and Node.js. Responsive means that you can not only query the current state, but also subscribe to all state changes, such as the results of a query or a single field of a document.

rxdb-realtime.gif

This is very useful for real-time UI-based applications because it is easy to develop and has great performance advantages. To replicate data between the client and server, RxDB provides modules for real-time replication with any CouchDB compatible endpoint and custom GraphQL endpoint.

RxDB supports the following features:

  • Mango-Query:Support mquery API to get data from collection, and support chained mongoDB query style.
  • Replication:Because RxDB depends on PouchDB, it is easy to synchronize data between the terminal device and the server.
  • Reactive:RxDB makes it easy to synchronize the state of the DOM.
  • MultiWindow/Tab:When two instances of RxDB use the same storage engine, their state and operation flow will be broadcast. This means that for two browser windows, the data change of window #1 will also automatically affect the data state of window #2.
  • Schema:through jsonschema to define Schemas, they are used to describe the data format.
  • Encryption:By setting the mode field to encrypted, the value of this field will be stored in encrypted mode and cannot be read without a password.

2.4 NeDB

The JavaScript Database, for Node.js, nw.js, electron and the browser.

https://github.com/louischatr...

NeDB is a JavaScript database that can run in Node.js, nw.js, Electron and browser environments. It is implemented using pure JavaScript and does not depend on other libraries. The provided API is a subset of the MongoDB API. The important thing is that it is very fast:

  • Insert:10,680 ops/s
  • Find:43,290 ops/s
  • Update:8,000 ops/s.
  • Delete:11,750 ops/s.

ops(operation per second) means the number of operations per second.

2.5 Dexie.js

A Minimalistic Wrapper for IndexedDB.

https://github.com/dfahlander...

Dexie.js is a packaging library for IndexedDB, which provides a well-designed API, powerful error handling, and strong scalability, in addition It can track data changes and supports KeyRange(search is not case sensitive, can be set to match mode and OR operation).

Dexie.js is mainly to solve the three main problems in the native IndexedDB API:

  • Abnormal error handling.
  • Weak query function.
  • Code complexity.

In order to facilitate developers to access Dexie.js, Dexie.js official website provides a wealth of examples:

  • React + Dexie
  • React + Redux + Dexie
  • Dexie with Typescript
  • Angular + Dexie
  • Dexie with Electron
  • Full Text Search

Only some examples are listed above. For more examples, please visit: Dexie.js-Samples ( https://dexie.org/docs/Samples ). Finally, let's briefly introduce various Web storage solutions.

3. Introduction to various web storage solutions

HTTP Cookie(also called Web Cookie or browser cookie) is a small piece of data that the server sends to the user's browser and saves it locally, it will be carried and sent to the server when the browser next initiates a request to the same server . Generally, it is used to inform the server whether two requests come from the same browser, such as keeping the user's login status.

Cookies are mainly used in the following three aspects:

  • Session status management(such as user login status, shopping cart, game score or other information to be recorded);
  • Personalized settings(such as user-defined settings, themes, etc.);
  • Browser behavior tracking(such as tracking and analyzing user behavior, etc.).

Features of cookies:

  • Cookie size is limited, generally 4 KB;
  • The number of cookies stored under the same domain name is limited, and the number of different browsers is different, generally 20;
  • Cookie support to set expiration time, when the expiration time is automatically destroyed;
  • Every time an HTTP request under the same domain is initiated, the cookie under the current domain name will be carried;
  • Support setting to HttpOnly to prevent cookies from being accessed by client's JavaScript.

Example 1:Simple usage

document.cookie = "name=semlinker";
document.cookie = "favorite_food=tripe";

alert(document.cookie);
//Show:name=semlinker;favorite_food=tripe

Example 2:Get cookie named test2

document.cookie = "test1=Hello";
document.cookie = "test2=World";

var myCookie = document.cookie
    .replace(/(?:(?:^|.*;\s*)test2\s*\=\s*([^;]*).*$)|^.*$/, "$1") ;
alert(myCookie);

3.2 localStorage

A persistent storage method, which means that if you do not manually clear, the data will never expire. It uses key-value pairs to store data, and saves the data to corresponding database files by domain name. Compared with cookies, it can save larger data.

Features of localStorage:

  • The size limit is 5MB ~ 10MB;
  • Share data between all tab pages and windows of the same source;
  • The data is only saved on the client, and does not communicate with the server;
  • The data persists and does not expire, and still exists after restarting the browser;
  • The operation of the data is synchronized.

Example

//Add a data item through setItem()
localStorage.setItem('myName','Semlinker');

//Get a data item through getItem()
let me = localStorage.getItem('myName');

//Remove a data item through removeItem()
localStorage.removeItem('myName');

//remove all data items
localStorage.clear();

3.3 sessionStorage

Similar to the server-side session, sessionStorage is a session-level cache, and the data is cleared when the browser is closed. It should be noted that the scope of sessionStorage is window level, which means that sessionStorage data saved between different windows cannot be shared.

Features of sessionStorage:

  • The data of sessionStorage only exists in the tab page of the current browser;
  • The data still exists after the page is refreshed, but the data will be cleared after closing the browser tab;
  • Have a unified API interface with localStorage;
  • The operation of the data is synchronized.

Example

//Add a data item through setItem()
sessionStorage.setItem('myName','Semlinker');

//Get a data item through getItem()
let me = sessionStorage.getItem('myName');

//Remove a data item through removeItem()
sessionStorage.removeItem('myName');

//remove all data items
sessionStorage.clear();

3.4 Web SQL

The Web SQL database API is actually not part of the HTML5 specification, but a separate specification that introduces a set of APIs to use SQL to operate the client database. It should be noted that HTML5 has abandoned the Web SQL database.

Three core methods defined in the Web SQL Database specification:

  • openDatabase:This method uses an existing database or a new database to create database objects;
  • transaction:This method allows us to control the commit or rollback of the transaction according to the situation;
  • executeSql:This method is used to execute real SQL statements.

Features of Web SQL(compared to Cookie, localStorage and sessionStorage):

  • Web SQL can facilitate object storage;
  • Web SQL supports transactions and can easily perform data query and data processing operations.

Example

var db = openDatabase('mydb', '1.0','Test DB', 2 * 1024 * 1024);

db.transaction(function(tx) {
   //Perform query operation
   tx.executeSql('CREATE TABLE IF NOT EXISTS LOGS(id unique, log)');
   //Perform the insert operation
   tx.executeSql('INSERT INTO LOGS(id, log) VALUES(1, "foobar")');
   tx.executeSql('INSERT INTO LOGS(id, log) VALUES(2, "logmsg")');
});

3.5 IndexedDB

IndexedDB is a low-level API for clients to store large amounts of structured data, including files and binary large objects. The API uses indexes to achieve high-performance searches of the data. Although Web Storage is useful for storing smaller amounts of data, it is not very useful for storing larger amounts of structured data. IndexedDB provides a solution.

IndexedDB features:

  • Large storage space:storage space can reach hundreds of megabytes or more;
  • Support binary storage:it can not only store character strings, but also store binary data;
  • IndexedDB has the same source restriction, each database can only be accessed under its own domain name, not across domain names;
  • Support transaction type:The operations performed by IndexedDB will be grouped according to the transaction. In a transaction, either all operations will succeed or all operations will fail;
  • Key-value pair storage:IndexedDB uses an object store to store data. All types of data can be stored directly, including JavaScript objects. In the object warehouse, data is stored in the form of "key-value pairs", and each data record has a corresponding primary key. The primary key is unique and cannot be repeated, otherwise an error will be thrown.
  • Data operations are asynchronous:operations performed using IndexedDB are performed asynchronously to avoid blocking applications.

Example

var dbName = "my_db";

var request = indexedDB.open(dbName, 2);

request.onerror = function(event) {
  //Error handling
};

request.onupgradeneeded = function(event) {
  var db = event.target.result;

  //Build an object warehouse to store information about our customers, we choose ssn as the key path
  //Because ssn can be guaranteed not to be repeated
  var objectStore = db.createObjectStore("customers", {keyPath:"ssn" });

  //Create an index to search for customers by name. The name may be duplicated, so we cannot use unique index
  objectStore.createIndex("name", "name", {unique:false });

  //Use the mailbox to build the index, we ensure that the customer's mailbox will not be duplicated, so we use the unique index
  objectStore.createIndex("email", "email", {unique:true });

  //Use the transaction's oncomplete event to ensure that the object warehouse has been created before inserting data
  objectStore.transaction.oncomplete = function(event) {
    //Save the data to the newly created object warehouse
    var customerObjectStore = db.transaction("customers", "readwrite").objectStore("customers");
    customerData.forEach(function(customer) {
      customerObjectStore.add(customer);
    });
  };
};

Space is limited. Here we only introduce some open source libraries. In fact, there are some other mature open source libraries, such as lowdb(Local JSON Database), Lovefield(Relational Database), and LokiJS(NoSQL Database). If you know other fun projects, Welcome to leave a message to Brother A Bao.

4. Reference Resources