PWAs
Progressive Web Applications add functionality to websites running in a web browser, to make them behave like a traditional software application.
Why
There are many ways a website can offer a better experience for people without having to install an application. It just comes down to...
Characteristics:
Progressive: Work for a variety of web browsers and when the user's browser supports modern functionality, it's taken advantage of. Responsive: Work well on mobile phones, tablets, and PCs. Works Offline: Work regardless of flaky network connections. App-Like: Can be installed to home screens, and be launched possibly without looking like a web browser. Fresh: Should get latest updates when available, without a complicated installation process. Safe: Certain features require HTTPS to operate securely Discoverable: Should have rich metadata associated with different URLs that are included in search engine results for the website's pages. Re-Engageable: Should allow easy re-engagement with users through notifications. Durable: Should be able to store data that lasts beyond a browser tab's lifetime. Beyond cookies and localStorage. Linkable: Maintain links between open web and installed applications.
But...
If you want to add additional features, one should use metrics to ensure they don't negatively impact performance or user experience. Include:
Time to First Paint Time to Interactive Time to Showing Dynamic Loaded Data Gzipped app size
Chrome DevTools
Network tab: Filter by XHR (network requests), and look at: Names column. Items can further be filtered using string literals or regular expressions. Status Code, Protocol (http1, http2), Size (indicators of service workers usage), Time, Waterfall. Performance Tab: Timeline (how hard the CPU is working, memory used, rendering v. computation using CPU) Network Requests (compared to Timeline) Flame Charts (how much time each function takes). Since we aspire our web apps to work at 60 fps, allowing us avg. 13ms per function invocation, before jank happens. All because JS is single-threaded.
Devices Mode: Simulate device characteristics like brand/model, dimensions, geolocation, orientation, touch... See how responsive breakpoints affect layout
Application Tab: Includes data on manifest, service workers, storage. When making dev updates, clear our existing service workers to refresh browser data.
Lighthouse: Chrome Plugin Measures performance, examines metadata, what's missing, give suggestions and overall scores. Requires serving over http2, since webpack dev server doesn't redirect to serve over https. But http2 does.
Additional Tools
WebPageTest.org Allows running speed tests from various locations around the world, using real browsers at actual connection speeds. Supports advanced features including: multi-step transactions, video capturing, single point of failure testing, content blocking, scripting.
Simulators Using Xcode or Android Studio, where localhost is your machine. Critical for testing features specific to either platform or browser. Also possible connect a real device to the dev machine with an USB cable for actual testing, linked to a browser (on PC) for debugging with breakpoints.
Progressive Metadata
Viewport <meta name="key" content="value"> (apple): viewport: width=device-width, initial-scale=1, user-scalable=no Fullscreeen <meta name="key" content="value"> apple-mobile-web-capable: yes apple-mobile-web-app-status-bar-style: black or white (battery bar at top of screen) apple-mobile-web-app-title: application title (home screen)
Manifests At least one, but perhaps up to 3-4 different manifests depending on PWA implementation Primary Manifest.json (or other name) <link rel="manifest" href="/manifest.json"> Part of Progressive Web App Standard Show example code For Icons, include home screen, and splash screen. Theme color: for address bar if browser supports it Background color: If HTML body has no background-color, this is the underlying default. Also for bg in pull-to-refresh area. display: (standalone and browser most supported). Degrades down to next display mode if unsupported. - fullscreen -- All available display area used - standalone -- Look/feel like standalone app. Close as we can get in iOS. - minimal-ui -- Doesn't have its own window; includes perhaps just back and share buttons. - browser -- like regular web browser Gotchas: remove any comments in json file.
Home Screen Icons: <link href="location" sizes="152x152" rel="apple-touch-icon"> different sizes and rel values for different devices. If size doesn't correspond exactly to device's needs, it'll act as if one didn't provide any image at all. However more and more, the manifest.json is used as the definitive list instead of link tags. Fallback solution: github.com/cubiq/add-to-homescreen
Schema.org Metadata included in script tag, for giving context to components or individual pages. show script
Loading and Rendering
Ideally, common files will only be downloaded once, and only other time-unique data will be loaded on subsequent app loads, depending on the caching strategy. The goal is a nearly-instant boot time of under 2 seconds.
With a client rendered SPA, we have traditionally 4 entities with a User making contact to: Static File Host, then CDN for Asset Storage.
Diagram: (load) Index.html, app.css, (load then parse) app.js, boot. At this point we have a non-database tied 'first paint'
Then we hit the Application API, which sends json data back.
With an SPA, nothing will get painted until the JS is downloaded, parsed, and booted. While server-side rendering is viable, it's complicated and error-prone . It basically entails "your server is booting up, programmatically grabbing the HTML for your JS app. 1) Startup your JS app 2) Sent it to a URL and wait for all the JSON to come back to it 3) Grab the JSON and respond to that HTML ?????
Server-Side Rendering
"Dynamic HTML responses on a per-request basis Involves special treatment of browser-only concepts -- XMLHttpRequest -- window, navigator, sensors, localStorage, screenWidth, screenHeight Initial HTML response is slowed down... ...but when it arrives, URL-specific content is ready!"
There will be no static file hosting, but rather a rendering server for views corresponding to particular paths. Data in users' cookies may also be taken into account for customizing content and views. Example: "Remember what the user's window width used to be, put that in a cookie, and use that for subsequent requests as a starting point".
All of this "dramatically slows down the initial HTML response". "What's on the critical path to returning HTML? Booting your app, going and getting data, rendering all those components, waiting for everything to settle, scraping that HTML, building the response object, sending it back to the browser." All before JS is loaded.
The advantage is that the user only hits the rendering server, which talks to the application API (much faster than client-side requests), then sends response back. The JS is only then requested from asset CDNs, and the client's HTML is updated as relevant to the application. Additional data requests are then sent from the client straight to the application API.
This requires a sharp eye for using JS appropriate to the context, as the DOM API isn't understood on the server without using a transpiler or library for stubbing and DOM simulation.
Client-Side Rendering
This may beg the question "Why not just do a traditional server rendered app?".
A reasonable first approach is "Enhanced Client-Side Rendering", which builds a shell for the web app. This shell is a container with a user interface that works with various URLs and screens, along with scripts that load, build, and maintain a stable application on first load. This allows fewer network requests and faster startup times on subsequent loads. The performance timeline in browser dev tools helps measure improvements.
3 Steps: 1) "Boot the app with URL-agnostic HTML" 2) "Embed critical CSS in your index.html", and maybe a little embedded JS 3) "Add a loading spinner"
Benefits: No 2nd round trip to load/parse CSS for styling and basic interactivity, at least top-level navigation.
Resource Caching Strategies
Check with Server if a file in the browser cache should still be used "Content at a URL can change" One could use Cache-Control: "no-cache" or use Last-Modified or ETag Downside: Involves extra network requests
Immutable Content Unique IDs, timestamps, or checksum appended to filenames Content at a URL is never changed One could use max-age=year However this strategy depends on index.html as the source of truth, deciding on exactly what filenames to load/use. Yet, index.html can't be itself cached. Offline-friendly approach
Limitations "Timestamps and version change notifications are not enough for offline apps to work." The idea of doing things a certain way so my intent is most likely inferred, is weak. "There's no place for imperative logic, based on request particulars, environmental conditions, etc." "Appcache tried to solve this, but it's awful"
"New Offline" "Not a prescriptive solution this time = more like 'ingredients' for a solution" "Offline can be totally different things depending on your app, what parts should be offline, what parts fail if there's no network", what app data can be saved now locally and synchronized later? "High degree of customizability. We don't all gave the same needs!" "Built on existing modern web concepts that are broadly supported"
Enter Service Worker (LINK)
"Cache-Only" (perhaps least useful). "Will try to pull everything from cache and never get to the network. It's supposed to be 'offline-first' not 'offline-only'". For resources that practically never change like a logo or fallback image.
"Network-Only" What you'd get if you remove 'event.respondWith()'.
"Cache with a Network Backup" get from cache or continue fetch(event.request).
Precache Important items only and (if available) the ability to download a new version of your application logic, which will load up NEXT time the application is run. "Great trade off between being slightly out-of-date, but the price you pay for a nearly instant boot" (subsequent). With this strategy, you focus on the heaviest part of your app.
"Network with Cache Backup" Get the resource from the network, and if it isn't available, get from cache. "Great strategy to use with JSON get requests, in case new data can't be retrieved." Situations where this would be called: being offline, server timeouts, taking too long (developer defined setTimeout), certificate errors. But not a 404. For that, you'd need to check response statuses other than 200 (ok). This is for having things quickly (JSON responses, images, etc) available, rather than the most recent version. "Doesn't apply to JS which you'd want to have fresh most of the time".
NOTE: The more you refactor out fetch logic into individual functions, the more unit-testable the program is.
"Cache then Network" Fetch "very important" data from the network, then pass it to a function that decides what to do with it. Like check for certain conditions, cache certain items, get new items, do something else. If everything's okay, then the application continues along. If conditions fail or whatever other reason, you can return an error, handled in catch() by getting new data from the network.
Cache and Update Provide a cached item when requested, then checks if a new version is available and if so, downloads it and puts it in cache.
Cache, Update, and Refresh Same as above, adding the option to notify the application that a new version is available. And the application can react however is appropriate. This wouldn't be appropriate to do with JS because of potential application scope and state conflicts.
"Generic Fallback" If all else fails, get this generic item from cache.
Composition
How should the strategies be composed? Two things to utilize: 1) Ability to check the request url for certain patterns, comparing origin 2) Accept Header for checking resource types: 'image/*', 'text/css', 'text/html', '*/*'
Checking the accept headers at the beginning allows you to branch out to various branches of next actions.
"Different types of files in different parts of your application will have different needs". Advice: 1) "Precache assets you know about at build time" 2) "Cache on the fly and fallback to assets that come from perhaps a database. As you encounter them, store them for later." 3) "Have a bias toward making your app boot really quickly. It's okay to trade a little bit of freshness to get that." Speed matters. If your application is built to be backwards compatible, this won't be a problem unless there's a security or other critical issue.
Create different caches for different strategies. Organize into an object with keys corresponding to each strategy, so you can call them by namespace. "Oftentimes we want to match a request against a specific cache, then handle that appropriately".
Versioning your cache(s), allows when doing cleanup (during the activate stage), to find and remove only specific versions.
How much can you cache? "Your origin (domain name) is given a certain amount of free space to do what it wants with. That free space is shared between all origin storage: LocalStorage, IndexedDB, Filesystem, and of course Caches". It depends on the browser and device, but one can check using the Quota Management API. After an origin host has exhausted their storage threshold, all data will be wiped and the storage process can start anew. This means one has to design defensively (or reasonably) so that such data fallouts don't cause a problem, and behavior defaults to as if loading the app fresh.
You can only respond to requests ONCE. Thus you need to write multiple conditionals in an event.respondWith, and ensure any called external functions simply return values, which are then returned in the conditional, thus ending the respondWith chain.
"Precache for Quick Boot"
Using an asset manifest for dealing with what you know about at build time. 1- "In your install event handler, fetch the asset manifest, and then iterate over this object to 'precache' all of the assets it names. 2- In your activate event handler, iterate over all of the caches, and delete any that are no longer useful. 3- In your fetch event handler, try to serve these assets form the cache (if possible) and fall back to retrieving them from the internet.
3 Patterns of using Service Workers that "embody Progressive Enhancement", based on 3 categories of assets: -stuff you know about at build time -JSON -index.html
"Caching dynamic data"
Example app: "While our app is now able to boot without an internet connection, as soon as we need JSON or product images, we're in trouble if we're offline. -Implement a fetch event handler that implements the 'cache fallback' strategy for all GET requests that aren't Precached in Advance. --Core idea: use fresh data from a fetch if possible, and fall back to the cached data if something goes wrong. (ex: user is offline -Store these responses in a separate cache then the precached responses." Because there are occasional use cases for keeping fallback image cache over multiple versions of service worker. The dynamic data cache is better updated often, as it comes at no cost if we're already getting network data first anyway.
Strategy to be used: Network then Cache. Possible scenarios in successive order: 1- Successful response of fetched data -> proceed 2- Unsuccessful response 3- Network is down
Note: 1- Caught errors within the promise chain aren't useful, for security reasons as a normal user (non-developer) is not entitled to know why the request failed. 2- A 404 response is still a valid response, and needs to be handled appropriately by examining the status code. 3- If a group of data is requested but not found on the server, the response may be an empty array, which would need to be examined.
For the fetchApiDataWithFallback function, we'll return an opened fallback cache first, then fetch the request. With response bodies, we're only allowed to consume them once. So for subsequent response actions, as with "network then cache" strategy, we'll do a response.clone() into a variable. Then place it into the cache (cache.put(fetchEvent.request, clonedResponse)), and return the response. Note using cache.put instead of cache.add (which would make a NEW request and overwrite any corresponding value in the cache).
For any caught errors, we can return cached values with cache.match(fetchEvent.request).
"SPA treatment of index.html"
"SPAs have to pay special attention to index.html because some portion of the URLs path may be used for client-side routing. Precache index.html in your install handler, adding it to the prefetch cache.
const INDEX_HTML_PATH = '/'; const INDEX_HTML_URL = new URL(INDEX_HTML_PATH, self.location).toString()
"For local (same origin) GET requests that have the appropriate Accept header, attempt to fetch a fresh index.html, and fall back to the precached response if something goes wrong.
let isHTMLRequest = request.headers.get('accept').indexOf('text/html') !=== -1; let isLocal = new URL(request.url).origin === location.origin;
IndexedDB
For saving things on the client for later synchronization with a server. No longer about caching assets to provide baseline functionality, but rather long term data management and exchange. IndexedDB is a NoSQL db that lives in the browser with the following properties:
Versioned (particularly important for desktop apps). Say someone stores data in your app, disappears for 18 months then returns. You want to be able to still use that data, and perhaps upgrade the client data format to a future friendly format.
Indexable records, allowing categorization, grouping, sorting as needed.
Worker-friendly because it has an async API (unlike localStorage nor cookies).
Durable (like localStorage and cookies), and plays by the same rules of many of the webs storage solutions we're used to.
Supports values of many types (unlike localStorage/cookies). Data is kept in stores with corresponding defined document/object types.
Major Events
Open (name and version number) onupgradeneeded (creating and/or migrating the schema depending on the version number) onsuccess (if database initialized or found and ready to start transactions)
Libraries
Because the IndexedDB spec is older and without promises, various libraries exist to simplify usage. One popular recommendation is IDB.
Web Push
"Provide the ability to send messages to one or more of a user's browsers, with permission. The user will receive the message regardless of whether the app is open or in the foreground. Describes an open standard for a push service (that you don't have to worry about). Requires a service worker
Getting permission "gives you an object called push subscriptions. That has the info you need to prove that you're authorized to send notifications to that user's device.
"Then we take that PushSubscription, package it with a message we wish to send to browser, and send it to the PushService, which is responsible for finding that unique broweser and sending the message. The browser has to be open for it to work, but no particular tab. It's likely that in the future the browser runs background daemons to receive/react to browser notifications anytime, just like a native app can send notifications without being open.
"Asking the user for a PushSubscription Requires a ServiceWorkerRegistration object. Can happen at the time the SW is registered, or in a SW event handler. Requires same permission as Notifications. Setup subscribe options object, and then ask a 'pushManager' to subscribe to notifications using the options. This promise resolves to the push subscription.
Subscribe options: 'userVisibleOnly': true/false. Chrome may only allow true? 'applicationServerKey': known as a VAPID key
Voluntary Application Server Identification for Web Push "Nice for push services. Helps them distinguish between legitimate and bogus traffic (ex: DDOS attacks). You SHOULD use VAPID. Not about securing the rights to send a message to a user (that's the PushSubscription's purpose)".
Generating VAPID keys NPM library. Public key is used to generate an appropriate PushSubscription (in the browser). Private key is used to send a push message from your back end to the PushService
A PushSub looks like an object with an endpoint key (push server endpoint URL), and 'keys' key of public key / authentication secret. This is done behind the scenes for us.
Note: "Users may have more than one push sub PER BROWSER. You won't be notified about users 'unsubscribing' or 'expiring' until you attempt to send a push event. This is fine -- Browser vendors ensure your users get no more than one notification per brower.
Sending a Notification You don't want to do this without a backend library of some sort. Debugging problems in this process is profoundly painful. This pain is not unique to web push -- most services like these have poor developer visibility.
Receiving a Notification Just another event ('push') in your service worker to listen for. It's synchronous, not promise-based, and you can use/consume it more than once.
Notifications
Notifications can be triggered anywhere including from within service workers.
"Require users' permission; requesting permission returns a promise; permission is either 'granted' or 'denied' in the promise. You can check the permission later using Notification.permission, which will be 'default' if you haven't prompted the user.
Once denied, permission can't be requested again. So developers have good reason to explain notifications and benefits first before spontaneously asking the user using default browser request behavior.
States: "default" (not asked yet), "granted" (yes), "denied" (turned off forever). Closing the request window keeps the user in "default" state.
If permission is 'denied', other things can be done by simply conditionally checking for the status.
Probably want to do progressive enhancement via feature detection to "to make sure we're not calling methods on objects that don't exist"
If 'Notification' in window, if Notification.permission equals defaults, Notification.requestPermission().then with permission, next steps.
Notification constructor functions include a title message string and an options object, which can include a body, icon, and others depending on operating system and browser vendor.
Options in 3 main categories: 1- Visual 2- Behavioral 3- Informational
Visual
body (string, like a notification sub heading) icon (URL, a small image to go along with your notification) image(URL, larger image) badge (URL, only used on Chrome for Android) vibrate sound dir (for LTR or RTL, "auto" is default option)
Behavioral
tag (a short string, used for grouping notifications) data (any object you want to include) requireInteraction (boolean) renotify (boolean, forces the notification to stay visible) silent (boolean)
Informational ?
Background Sync
If we want to push data from the client to the network when there's momentarily no connection or data sent was dropped somewhere, we need to save it instead and sync when the network is back online.
"BS allows you to defer an action until you have a successful connection again. You can schedule an action. Your service worker will periodically try it out. If it works, great. If not, try again later.
Register sync events by name, listen for sync events, conditionally act based on event.tag's name using event.waitUntil.
If the sync event promise rejects, the sync event is still valid. Additional sync events with same title write over older ones, preventing duplicate events.
Considerations: "You can't pass in any state when you register your background sync. IndexedDB is a great choice as a place to store this data. When SW attempts a sync, read the data out of IndexedDB.
Background Sync Strategies
"Just like caching, there is no one-size-fits-all solution for syncing. In some situations, we might only want to schedule a sync if the network fails. In other situations, this might be our first choice"
HTTP/2
Upgrade to the HTTP transport layer Fully multiplexed-send multiple requests in parallel Allows servers to proactively push responses into client caches.
Why upgrade? Websites are growing: more images, more JS While bandwidth has improved, roundtrip time hasn't It takes just as long to ping a server now as it did 20 years ago. One file at a time per connection.
Workaround hacks
When having large numbers of images load, would be to create sprites (one load), then use background-positioning to show the correct part of the large image to show. One shouldn't have to pay extra bandwidth to lower the number of requests by using larger images and files.
Reusing connections: "Having many things hosted on a single domain so you can establish that TLS connection and get the DNS resolution out of the way, then hit that a few times.
Modern browsers will allow 6-9 simultaneous connections per domain. Using domain sharding allows distributing assets from multiple servers.
What's not different with HTTP/2? Same HTTP methods, headers, paths, semantics.
What's different? -"Multiplexing: multiple files can be sent simultaneously over a single connection. -A well configured server can know about what other files you will need based on the one you asked for and start sending it to you before you even ask".
Support If anything in the network doesn't support HTTP/2 the rest of the data flow (in that direction) will be dropped to HTTP 1.1. Today, devices, edge network tech, and CDNs support it. It's up to developers to cleverly organize their assets and provide header clues so that CDNs will apply HTTP/2
Patterns for PWAs
Application Shell
Architectural pattern based on the idea that your application has two distinct components: 1- UI elements 2- Content displayed
Makes PWAs more like native apps with core UI and just loading data from network. Tip: cache the application shell using a service worker. So even if we're online, we'll render that shell first before going to the network to get content. For the index.html stuff, we'll hit the cache first and only go to the network if the cache is empty.
In order for this to work, we need a way to get new versions index.html in the background.
PRPL
Push critical resources for the route that is being requested. This is where code splitting, route splitting, and http/2 push is really important. Get these assets over the wire in parallel asap. Render the initial route Pre-cache the remaining routes and application fragments in the background so they're available instantly. This is akin to using pre-caching using asset-manifests. Lazy-load the remaining routes when they are ready. "You'll want to defer evaluating those scripts because there is some cost for taking text JS files and turning them into executable code. There's no need to pay that cost until you're ready to use them. So you can keep them in the cache API where they're inert, as they're just text. Then as you need them, you can send a request for them, the service worker will send them back to the application, at which point it becomes code, gets evaluated, and is usable".
Goals: 1- Minimize the time-to-interactive by "shrinking the code we send to the browser for a given URL, shrinking the CSS (so we're not sending all of Twitter Bootstrap along).
2- Maximize caching efficiency "by trying to ensure we have the most critical stuff first so that if a user visits a page, and switches to another one, you have what's needed first and the rest comes later".
Offline Data
Case
Flaky Connections Users switch off data to conserve it in small towns/villages in developing world
Implementations
Service Workers with Cache API - Straightforward for read-only offline mode Probably using IndexedDB with libraries
However when writes are more prevalent and for a closer integration (when you're expecting more offline situations), consider how the data model can be persisted. Redux Stores for example can be modified to be persistent.
The ReduxOffline lib for example persists a redux store to disk on every change automatically reloads on startup, creates an offline subtree of logged actions, and manages this subtree. When the User if online again, Redux reapplies the actions in the subtree. One needs to be careful though because it's in-memory. ???
Getting server-side persistence and quicker offline-online sync. Options: PouchDB Enables local data storage when offline (IndexedDB for web apps, LevelDB for node apps) Syncing with servers once online (CouchDB inspired)
Considerations
Data Validation - It can be a huge waste of time for users if state is updated offline, only to be invalidated later when online because that's where all the validation logic exists.
"Concurrent Data Edits" and sync (if applicable to product)
How syncing on/offline is communicated with users
Data versioning to determine what is older v most current
Data Conflict resolutions
Offline Storage Space (dependent on browser and device) Quota Management API (with increasing support), should help.
Last updated