Firestore 

using with 

Web and WebViewer Components

INTRO

Before you start in on this, I will recommend that you have a good read and watch the videos, provided in the RESOURCES at the end of this page, and check out any other pages from the official documentation that you feel you may need to read, then come back here. Pay particular attention to the Firestore pricing page, even though you will start off with the free part of the Spark plan, it really does not take much use of Firestore to get to the free 50,000 reads limit!

Google touts Firestore, its successor to the Realtime Database, as the go to No-SQL scalable database. It is quite a different beast to the Realtime Database, and it will take a little time for you to get your head around working with "collections, documents and fields", but after that it starts to make sense, then using, working with, and querying the database becomes more straightforward.

In this guide, I will look at two main approaches to using Firestore with Appinventor blocks: with the Web component, and with the Webviewer. Some methods you can only use with one component, most you can use with both, but using them together you can build an app that interacts with Firestore, and helps you to minimise your "reads and writes" . I use the Firestore REST API to interact with the database. The main drawback is that, for the most part, Google and Firestore are expecting you to use a recognised framework or Android Studio to build your app.  The use of the REST API means that you have to create your own "local cache" of already downloaded data (this is the thing that will save you on "reads and writes", along with the queries you write). For the "local cache" I will use the tinyDB, which is more than up to the task.

I will not be covering Firebase Storage or Security Rules in any great depth here, you can find more information about that in my other guide. I will touch on Authentication at some point, but for the most part will be running Firestore with "read =true / write=true".

A very brief explanation of the Firestore data structure:

Firestore uses collections, documents and fields. Think of a collection as a folder. You can then add one or more documents to a folder. Each document can contain fields (key:value pairs - data/content). Documents can contain sub collections (so a folder inside the document), and these sub collections can also contain documents with fields. This way you can build up a "tree" structure for your data.

A simple example of a chat app:

Create a root collection called chats

In chats create a document called chatroom1 with a field {"name":"roomA"}.

In chatroom1 create a sub-collection called messages

In messages create documents containing each chat message -

 e.g. msg1 with fields {"name":"Joe", "message": "Hi there", "timestamp": 1626478921234"}

In order to view the message just sent you would need a path like so:

GET  chats/chatroom1/messages/msg1

You can give collections and documents names, or let firestore generate an autoID. The timestamp field can be important in your data, because Firestore autoIDs are not lexicographical or numerical , you will need a timestamp in order to query effectively. You should be able to see that under chats, you could have many chatroom1s, each with their own messages and msg1s

I will use this approach to demonstrate the blocks/code.

SETUP

Setup a Firestore Project

Let us assume you already have a Firebase account, go to this url to set up your Firestore:

https://firebase.google.com/docs/firestore/quickstart

Put together all your configuration resources

Go to the Project Overview/ Project Settings and grab the Firestore Config details, you will need some/all of these later:
(dummy data)

const firebaseConfig = {

  apiKey: "AIzbZwCaDWoy-zUODIcn9o5F6y3tCC2VvfJPDuk",

  authDomain: "firestoredemo-1234f.firebaseapp.com",

  databaseURL: "https://firestoredemo-1234f.firebaseio.com",

  projectId: "firestoredemo-1234f",

  storageBucket: "firestoredemo-1234f.appspot.com",

  messagingSenderId: "1234567891011",

  appId: "1:1123456789:web:db0dg34a1d4dd1355b97c3",

  measurementId: "K-58UJ6HZJHC"

};

The most useful is the <projectId>. Keep this safe, while you are running without authentication!

For the purposes of this demo, I have my rules setup like this

rules_version = '2';
service cloud.firestore {

  match /databases/{database}/documents {

    match /{document=**} {

      allow read, write: if request.time < timestamp.date(2022, 10, 19);

    }

  }

}

which should stop Google/Firestore pestering you about authentication, until you get close to the date set....

Setup your App Inventor App

For your app, as a minimum you will need to drag in the Web component and a Webviewer (which does not have to be visible), and buttons/arrangements/labels for actions and values. I also used Juan Antonio's JSON Decode extension to help extract the fields, when using blocks to return data. I generally just connect to companion and run the code direct from the blocks editor, and set a label to return the output. 

When working with blocks, you will need to use the same url over and over again, it is therefore best to create a variable for this:

urlPath = https://firestore.googleapis.com/v1/projects/firestoredemo-1234f/databases/(default)/documents/

BUILD

OK, here we go... I am going to do everything in simple steps, but you will find it is possible to create collections/documents in a chain on the fly.

CREATE

Let us create our root collection (a collection cannot be empty, you have to create a document as well):

You should see that Web1.GotText has returned:

{
  "name": "projects/firestoredemo-1234f/databases/(default)/documents/chats/chatroom1",
  "fields": {
    "chatroom": {
      "stringValue": "RoomA"
    }
  },
  "createTime": "2021-11-14T11:04:49.689163Z",
  "updateTime": "2021-11-14T11:04:49.689163Z"
}

and if you look in your Firestore console you will see:

You can extract the field data using the following blocks:

The above will return as: RoomA

Now we can create a message. After this a slight modification is needed to these blocks to continue to create new messages in our app.

Web1.Gottext should have returned this:

{
  "name": "projects/firestoredemo-1234f/databases/(default)/documents/chats/chatroom1/messages/msg1",
  "fields": {
    "name": {
      "stringValue": "Joe"
    },
    "message": {
      "stringValue": "Hello World"
    },
    "timestamp": {
      "integerValue": "1636890418602"
    }
  },
  "createTime": "2021-11-14T11:47:00.753869Z",
  "updateTime": "2021-11-14T11:47:00.753869Z"
}

You should now see this in your Firestore console. (note the firestore console does not always automatically update when a change is made (because this generates a "read") so you may have to refresh your browser to see added data)

and you can extract the message fields data like so:

From now on you can create new messages using the block structure below. I have used some variables in the procedure, so all you have to do is fill in the blanks! Notice that I do not set a name for any message itself (/messages/). I get Firestore to "autoID" the individual message names to save time and effort, and ensures that messages sent by any user get a unique name.

This then returns the following with Web1.GotText, you can see the autoID message name...

{
  "name": "projects/firestoredemo-1234f/databases/(default)/documents/chats/chatroom1/messages/DC9z2pQpdbFjWKJ8Ja5n",
  "fields": {
    "name": {
      "stringValue": "Bob"
    },
    "timestamp": {
      "integerValue": "1636893347485"
    },
    "message": {
      "stringValue": "Hi there"
    }
  },
  "createTime": "2021-11-14T12:35:49.685657Z",
  "updateTime": "2021-11-14T12:35:49.685657Z"
}

OK, that just about covers "creating" collections and documents, sub collections etc.  I am going to keep going on about reads and quotas throughout, just putting this little bit together on Firestore took me 5 writes and 98 reads.

GET Data

This falls into two categories, a straightforward http GET, and QUERIES. Let us look at GET first. You need to provide the correct path in order to return the data correctly.

Get a single document will return the fields for that document

Get a collection will return all the documents and their fields

QUERIES - building a query for the first time with the blocks is not a lot of fun...getting the syntax correct is the most difficult part, switching between dictionaries and lists to create the correct brackets etc. However, once you have your base structure set up, you can easily modify your query to return what is required for your app. in the example below I will query all messages submitted AFTER a specific timestamp and limit the returns to 25. The will be returned in order, descending from the latest message. As with GET, you must provide a full path the to collection or documents you wish to query (but see from the blocks that the messages collection is "inside" the query).

Here is the top line for the query:
https://firestore.googleapis.com/v1/projects/firestoredemo-1234f/databases/(default)/documents/chats/chatroom1:runQuery

and the query syntax in json:

{"structuredQuery":{

"from":[{"collectionId":"messages"}],

"orderBy":[{"field":{"fieldPath":"timestamp"},"direction":"DESCENDING"}],

"where":{"fieldFilter":{"field":{"fieldPath":"timestamp"},"op":"GREATER_THAN","value":{"integerValue":1636884000000}}},

"limit":25}}

and the blocks for the query:

which return this output to the Web1.GotText(I only have two messages...):

I have only scratched the surface of what you can do with queries. You can write more complex queries across collections, this requires the use of a composite filter.


EDIT ( fields)

We now have the PATCH command in the Web component, which allows us to edit/update individual fields in a document. Do not use POST for this, it will delete all the fields in a document apart from the field you are updating, use PATCH. Make certain that you include the update Mask parameter, because this selects the field to be edited/created. Let us edit's Bob's message from "Hi there" to "Howdy!". You will need to capture the individual message ID.

Add a New Field to a Document (this will do nothing other than return the field data, if the field and the same content already exist)

DELETE

You can delete fields in a document, documents and collections. The firestore documentation indicates that deleting a top/higher level collection/document does not necessarily delete any sub-collections within it, so you may need to work from the bottom up when deleting such things.

Delete a Single Field in a Document

Deleting a Document

Delete a Collection

First get a list of all the documents in the collection. Delete each of these. When the last document is deleted, the collection will also be deleted.


REALTIME UPDATES

This is where the magic can happen - if you want it to... Some apps can really benefit from real time updates, e.g. a chat app, whereas for other apps it maybe that you only want to update the data when a user chooses, or when it is sensible for the app to do so automatically. The Firestore video guy goes on about this quite a bit in Firestore Video #10.

You cannot get real time updates using the REST API, you will have to build a little one page web app in html and link this up with the App Inventor webviewstring to send query details and receive the returned data. Here is an example of what is needed, again using the chat app as an example. We do much the same with the query as was written for the blocks version, I send the path to the messages collection (via the webviewstring), set a timestamp to select the matching documents, order them in descending order, limit the number returned, and return the data back through the webviewstring.

The initial webviewstring would be constructed something like this:

1636884000000,chats,chatroom1,messages

The data is returned in a json list of lists, which is much easier to handle than in the form returned by the blocks. The first time you setup the listener, you will get ALL matching documents returned, after that, only newer documents are returned, automagically ;)

I have not yet found an automatic method to stop the listener when a user leaves the app (to keep down reads in the background), this is because the listener is set inside the html when it first runs, and you have to be in the html in order to stop it. A simple page refresh will not work. In a demo app I have been working on, I provide the user with a start / stop button in the html, and then in App Inventor pop up notifiers if they choose to leave/close the app to remind them to "unlisten" if they have not done so. 

This is also where the local cache and tinydb come into play. I store the returned results on first run in tinydb, then add new incoming results to the tinydb list. On an app restart, the local cache is compared to the new results (using timestamps) and the correct results are displayed.

Finally I have a routine setup to delete documents older than 90 days (not fully tested) in the html, and a reciprocal routine setup for the local cache. This keeps down the data load on Firestore / App Inventor (but only useful for things like chat apps...)

<head>

  <title>FB-FIRESTORE-AI2</title>

  <script src="https://www.gstatic.com/firebasejs/8.10.0/firebase-app.js"></script>

    <script src="https://www.gstatic.com/firebasejs/8.10.0/firebase-firestore.js"></script>

    <script src="https://www.gstatic.com/firebasejs/8.10.0/firebase-auth.js"></script>

</head>

<body>

<button id="listen" onclick="Listen()" style="background-color:green;color:orange;width:80px;height:40px;border:none;outline:none;border-radius:5px"><b>Listen</b></button>

<script>

//get webviewstring list from app

var wvstr = window.AppInventor.getWebViewString();


//set the firebase configuration

var firebaseConfig = {

  apiKey: "AIzbZwCaDWoy-zUODIcn9o5F6y3tCC2VvfJPDuk",

  authDomain: "firestoredemo-1234f.firebaseapp.com",

  databaseURL: "https://firestoredemo-1234f.firebaseio.com",

  projectId: "firestoredemo-1234f",

  storageBucket: "firestoredemo-1234f.appspot.com",

};


var readts = parseInt(wvstr.split(",")[0]);

var mycoll = wvstr.split(",")[1];

var mydoc  = wvstr.split(",")[2];

var mysub  = wvstr.split(",")[3];

var delts  = parseInt(wvstr.split(",")[4]);

var fblistener = null;


//initialise firebase and firestore reference

firebase.initializeApp(firebaseConfig);

var db = firebase.firestore();


function Listen() {

if ( document.getElementById("listen").style.backgroundColor == "green") {

document.getElementById("listen").style.backgroundColor = "red";

document.getElementById("listen").innerHTML = "<b>UnListen</b>";

document.getElementById("listen").style.color = "white";

fblistener = db.collection(mycoll).doc(mydoc).collection(mysub).where("timestamp", ">",            readts).orderBy("timestamp", "desc").limit(25)

.onSnapshot((querySnapshot) => {

var myData = [];

querySnapshot.forEach((doc) => {

                myData.push([doc.data().from, doc.data().message, doc.data().timestamp]);

});

readts = parseInt(myData[0][2]);

delts = (readts - 7900000000);

window.AppInventor.setWebViewString(JSON.stringify(myData));

});


deleteOldDocs();


} else {

document.getElementById("listen").style.backgroundColor = "green";

document.getElementById("listen").innerHTML = "<b>Listen</b>";

document.getElementById("listen").style.color = "white";

fblistener();

window.AppInventor.setWebViewString("listener stopped");

}

};


function deleteOldDocs() {

       var counter = 0;

    db.collection(mycoll).doc(mydoc).collection(mysub).where("timestamp", "<", delts).get()

    .then(function(querySnapshot) {

            var batch = db.batch();

            querySnapshot.forEach(function(doc) {

            if (counter !> 499 ) {

            batch.delete(doc.ref);

            counter += 1;

}

});

            return.batch.commit();

}).then(function() {

    }); 

}


</script>

</body>


</html>

SECURITY

Using Firestore with Firebase Authentication

You can use all the above with authenticated users and more secure rules. You will need to capture the idToken for the authenticated user when they sign in through the app (see here for all that), then use the token in an Authorization header for each call, as follows:

not sure what happens with real time updates in the html, will have to test out and report back

Data in App Inventor

For the purposes of this demo,  I have left security fairly lax. In the real world, the <projectId> needs obfuscating in your blocks. The firebase config data in the html file also needs some protection, you can pass the <projectId> and a part of the API key via the webviewstring, obfuscating those items in your app.

CONCLUSION

That just about covers things for now. This is a fairly basic get you started guide on using Firestore with App Inventor. Firestore is a big subject, there is alot you can do with it.

Back to reads and writes again. In the compilation of this guide, starting from scratch and testing each method, I generated 265 reads, 22 writes and 7 deletes. The issue will always be with the reads, and on this basis, it would only take @ another 180 users doing much the same light work as me to reach the daily quota of 50,000. You need to keep an eye on your usage as your app becoming more successful, and have a plan in place to handle your growth.