resolve conflicts between rebrand and staging

CHANGELOG.md

@@ -1,3 +1,96 @@
+# ZyzomysPedunculatus (3.25.0)
+
+## Goals
+
+This is the last major release of our 3.0.0 release cycle. We wanted to mark the occasion with some big improvements to keep everyone happy in case we need to take some more time to prepare our upcoming 4.0.0 release.
+
+## Update notes
+
+This update introduces some major database optimizations that should decrease both CPU and disk usage over time as users request resources and prime an on-disk cache for the next time.
+
+We've also introduce the ability to archive illegal or otherwise objectionable material from the admin panel assuming you possess the ability to load the content in question. It's also possible to restore archived content via an adjacent form field on the admin panel as long as it has not been permanently deleted. Due to a quirk in how ownership of uploaded files works, restored files will not retain their "owners" property. We hope to fix this in a future release.
+
+We've also made some minor changes to the example NGINX config file provided in `cryptpad/docs/example.nginx.confg`, specifically in [this commit](https://github.com/xwiki-labs/cryptpad/commit/2647acbb78643e651b71d2d4f74c2f66e264a258). CryptPad will probably work if you don't apply these changes to your nginx conf, but some functional improvements depend on the exposed headers.
+
+To upgrade from 3.24.0 to 3.25.0:
+
+1. Update your NGINX config as mentioned above.
+2. Stop your nodejs server.
+3. Pull the latest code using git (from the `3.25.0` tag or the `main` branch)
+4. Ensure you have the latest clientside and serverside dependencies with `bower update` and `npm install`.
+5. Restart  the nodejs server.
+
+## Features
+
+* This release makes a lot of changes to how content is loaded over the network.
+  * Most notably, CryptPad now employs a client-side cache based on the the _indexedDB API_. Browsers that support this functionality will opportunistically store messages in a local cache for the next time they need them. This should make a considerable difference in how quickly you're able to load a pad, particularly if you accessing the server over a low-bandwidth network.
+  * Uploaded files (images, PDFs, etc.) are also cached in a similar way. Once you'd loaded an asset, your client will prefer to load its local copy instead of the server.
+  * We've updated the code for our _full drive backup_ functionality so that it uses the local cache to load files more quickly. In addition to this, backing up the contents of your drive will also populate the cache as though you had loaded your documents in the normal fashion. This cache will persist until it is invalidated (due to the authoritative document having been deleted or had its history trimmed) or until you have logged out.
+  * We've added the ability to configure the maximum size for automatically downloaded files. Any encrypted files that are above this size will instead require manual interaction to begin downloading. Files that are larger than this limit which are already loaded in your cache will still be automatically displayed.
+* We've also changed a lot of the UI related to encrypted file uploads and downloads:
+  * Encrypted files can display buttons instead of the intended media under a variety of circumstances (if they are larger than your configured limit or if there is no applicable rendering mode). The styles for these buttons are now much more consistent with those found throughout the rest of the platform.
+  * The same assets should now display progress bars when downloading and decrypting encrypted media.
+  * When the same asset is embedded into a document in more than one location it used to be possible to trigger two (or more) concurrent decryption processes. We've modified the rendering process so that duplicates are detected and rendered simultaneously after the relevant assets have been decrypted (once).
+  * We noticed that some old code to filter out forbidden content from rich text pads was interfering with encrypted media. We've clarified the filtering rules to preserve such content (audio, video, iframes) when it occurs within an acceptable context.
+  * We've fixed some inconsistencies with media styles and functionality across different editors. Most types of media now allow you to right-click and choose to _share_ (open that asset's share menu) or open it in a different context (in the file app or in the relevant editor where this behaviour is supported).
+  * The _file_ app has been greatly simplified. It now uses the same methods to render encrypted media as is used elsewhere, so it also displays progress and has a more consistent UI.
+  * The file uploads/downloads table has also been improved somewhat:
+    * Download progress is displayed for groups of items when downloading a folder from your drive.
+    * We found and removed a hard-coded translation from the table's header.
+* In keeping with the theme of network traffic and files we've also made some improvements to policies for users' storage:
+  * Users should now be prompted to trim the history of very large documents when viewing them, saving space for the server operator as well as freeing up some of the user's quota.
+  * Users will also be prompted to use similar functionality available through the settings page when the history of their drive and other account-related functionality is consuming a significant amount of their quota.
+  * Documents that you own used to be automatically added to your drive when viewed if they weren't already present. This was originally intended as an integrity check and a means to recover from incorrectly removed entries in your drive, however, as we now support the removal of owned elements from your drive without destroying them this only serves as an annoyance. As such, we have dropped this functionality.
+  * The whiteboard editor allows users to insert encrypted images into whiteboards, but only up to a certain size. Before it would just warn you that your image was too large. Now it provides the actual size limit that you've exceeded.
+  * The prompt to store uploads in your drive is now suppressed when uploading images via the support ticket panel.
+
+## Bug fixes
+
+* This release includes a fix for a very severe bug in Chrome and its derivatives where attempting to open a URL from within our sandboxing system would crash the browser entirely. This version works around the problem by _not doing that_.
+* We've improved offline detection such that "offline" status is specific to particular resources like your drive, teams, and shared folders rather than treating your account as simply "online or offline".
+* We've optimized one of our less style sheet mixins that was used in a lot of places at a more specific scope than was necessary. This resulted in more time compiling styles and higher storage space requirements for the css cache in localStorage.
+* A small helper function that was intended to stop listening for `enter` and `esc` keypresses after closing a modal was overly zealous and stopped listening after _any keypress_. This made it so that any prompt with an input field did not correctly submit or cancel when pressing `enter` or `esc` after  typing some text.
+* Various browsers now require the request for the permission to send notifications to originate from a "click" event, so CryptPad now opens a dialog prompting you to allow (or disallow) permission if you haven't already made that decision.
+* Modern browsers commonly prevent tabs from opening new windows unless you've explicitly enabled that behaviour (it's an important feature), however, in some cases the indication that a new tab was blocked can be very subtle and some of our users did not notice it. We now check whether attempts to open a new tab were successful, and prompt the user to enable this behaviour so that CryptPad can perform regular actions like opening a pad from the drive.
+* After some deep investigation we identified a number of scenarios where contact requests would behave incorrectly, such as not triggering a notification. Contact requests should now be much more stable. On a related note, it's now possible to cancel a pending contact request from the concerned user's profile.
+
+# YunnanLakeNewt (3.24.0)
+
+## Goals
+
+We are once again working to develop some significant new features. This release is fairly small but includes some significant changes to detect and handle a variety of errors.
+
+## Update notes
+
+This release includes some minor corrections the recommended NGINX configuration supplied in `cryptpad/docs/example.nginx.conf`.
+
+To update from 3.23.2 to 3.24.0:
+
+1. Update your NGINX config to replicate the most recent changes and reload NGINX to apply them.
+2. Stop the nodejs server.
+3. Pull the latest code from the `3.24.0` tag or the `main` branch using `git`.
+4. Ensure you have the latest clientside and serverside dependencies with `bower update` and `npm install`.
+5. Restart the nodejs server.
+
+## Features
+
+* A variety of CryptPad's pages now feature a much-improved loading screen which provides a more informative account of what is being loaded. It also implements some generic error handling to detect and report when something has failed in a catastrophic way. This is intended to both inform users that the page is in a broken state as well as to improve the quality of the debugging information they can provide to us so that we can fix the underlying cause.
+* It is now possible to create spreadsheets from templates. Template functionality has existed for a long time in our other editors, however, OnlyOffice's architecture differs significantly and required the implementation of a wholly different system.
+* One user reported some confusion regarding the use of the Kanban app's _tag_ functionality. We've updated the UI to be a little more informative.
+* The "table of contents" in rich text pads now includes "anchors" created via the editor's toolbar.
+
+## Bug fixes
+
+* Recent changes to CryptPad's recommended CSP headers enabled Firefox to export spreadsheets to XLSX format, but they also triggered some regressions due to a number of incompatible APIs.
+  * Our usage of the `sessionStorage` for the purpose of passing important information to editors opened in a new tab stopped working. This meant that when you created a document in a folder, the resulting new tab would not receive the argument describing where it should be stored, and would instead save it to the default location. We've addressed this by replacing our usage of sessionStorage with a new format for passing the same arguments via the hash in the new document's URL.
+  * The `window.print` API also failed in a variety of cases. We've updated the relevant CSP headers to only be applied on the sheet editor (to support XSLX export) but allow printing elsewhere. We've also updated some print styles to provide more appealing results.
+* The table of contents available in rich text pads failed to scroll when there were a sufficient number of heading to flow beyond the length of the page. Now a scrollbar appears when necessary.
+* We discovered a number of cases where the presence of an allow list prevented some valid behaviour due to the server incorrectly concluding that users were not authenticated. We've improved the client's ability to detect these cases and re-authenticate when necessary.
+* We also found that when the server was under very heavy load some database queries were timing out because they were slow (but not stopped). We've addressed this to only terminate such queries if they have been entirely inactive for several minutes.
+* It was possible for "safe links" to include a mode ("edit" or "view") which did not match the rights of the user opening them. For example, if a user loaded a safe link with edit rights though they only had read-only access via their "viewer" role in a team. CryptPad will now recover from such cases and open the document with the closest set of access rights that they possess.
+* We found that the server query `"IS_NEW_PAD"` could return an error but that clients would incorrectly interpret such a response as a `false`. This has been corrected.
+* Finally, we've modified the "trash" UI for user and team drives such that when users attempt to empty their trash of owned shared folders they are prompted to remove the items or delete them from the server entirely, as they would be with other owned assets.
+
 # XerusDaamsi reloaded (3.23.2)
 
 A number of instance administrators reported issues following our 3.23.1 release. We suspect the issues were caused by applying the recommended update steps out of order which would result in the incorrect HTTP header values getting cached for the most recent version of a file. Since the most recently updated headers modified some security settings, this caused a catastrophic error on clients receiving the incorrect headers which caused them to fail to load under certain circumstances.

bower.json

@@ -30,7 +30,7 @@
     "secure-fabric.js": "secure-v1.7.9",
     "hyperjson": "~1.4.0",
     "chainpad-crypto": "^0.2.0",
-    "chainpad-listmap": "^0.9.0",
+    "chainpad-listmap": "^0.10.0",
     "chainpad": "^5.2.0",
     "file-saver": "1.3.1",
     "alertifyjs": "1.0.11",

config/config.example.js

@@ -42,7 +42,7 @@ module.exports = {
  *
  *  In a production instance this should be available ONLY over HTTPS
  *  using the default port for HTTPS (443) ie. https://cryptpad.fr
- *  In such a case this should be handled by NGINX, as documented in
+ *  In such a case this should be also handled by NGINX, as documented in
  *  cryptpad/docs/example.nginx.conf (see the $main_domain variable)
  *
  */
@@ -228,12 +228,12 @@ module.exports = {
      */
 /*
     customLimits: {
-        "https://my.awesome.website/user/#/1/cryptpad-user1/YZgXQxKR0Rcb6r6CmxHPdAGLVludrAF2lEnkbx1vVOo=": {
+        "[cryptpad-user1@my.awesome.website/YZgXQxKR0Rcb6r6CmxHPdAGLVludrAF2lEnkbx1vVOo=]": {
             limit: 20 * 1024 * 1024 * 1024,
             plan: 'insider',
             note: 'storage space donated by my.awesome.website'
         },
-        "https://my.awesome.website/user/#/1/cryptpad-user2/GdflkgdlkjeworijfkldfsdflkjeEAsdlEnkbx1vVOo=": {
+        "[cryptpad-user2@my.awesome.website/GdflkgdlkjeworijfkldfsdflkjeEAsdlEnkbx1vVOo=]": {
             limit: 10 * 1024 * 1024 * 1024,
             plan: 'insider',
             note: 'storage space donated by my.awesome.website'

customize.dist/ckeditor-contents.css

@@ -213,3 +213,61 @@ media-tag * {
     width: 100%;
     height: 100%;
 }
+media-tag button.btn {
+    background-color: #fff;
+    box-sizing: border-box;
+    outline: 0;
+    display: inline-flex;
+    align-items: center;
+    padding: 0 6px;
+    min-height: 36px;
+    line-height: 22px;
+    white-space: nowrap;
+    text-align: center;
+    text-transform: uppercase;
+    font-size: 14px;
+    text-decoration: none;
+    cursor: pointer;
+    border-radius: 0;
+    transition: none;
+    color: #3F4141;
+    border: 1px solid #3F4141;
+    max-width: 250px;
+}
+media-tag button.mediatag-download-btn {
+    flex-flow: column;
+    min-height: 38px;
+    justify-content: center;
+}
+media-tag button.mediatag-download-btn > span {
+    display: flex;
+    line-height: 1.5;
+    align-items: center;
+    justify-content: center;
+}
+media-tag button.mediatag-download-btn * {
+    width: auto;
+}
+media-tag button.mediatag-download-btn > span.mediatag-download-name {
+    max-width: 100%;
+}
+media-tag button.mediatag-download-btn > span.mediatag-download-name b {
+    white-space: nowrap;
+    overflow: hidden;
+    text-overflow: ellipsis;
+}
+
+media-tag button.btn:hover, media-tag button.btn:active, media-tag button.btn:focus {
+    background-color: #ccc;
+}
+media-tag button.btn b {
+    margin-left: 5px;
+    white-space: nowrap;
+    overflow: hidden;
+    text-overflow: ellipsis;
+}
+media-tag button.btn .fa {
+    display: inline;
+    margin-right: 5px;
+    flex: 0;
+}

customize.dist/loading.js

@@ -253,7 +253,7 @@ p.cp-password-info{
     animation-timing-function: cubic-bezier(.6,0.15,0.4,0.85);
 }
 
-button.primary{
+button:not(.btn).primary{
     border: 1px solid #4591c4;
     padding: 8px 12px;
     text-transform: uppercase;
@@ -262,7 +262,7 @@ button.primary{
     font-weight: bold;
 }
 
-button.primary:hover{
+button:not(.btn).primary:hover{
     background-color: rgb(52, 118, 162);
 }
 
@@ -291,7 +291,7 @@ button.primary:hover{
     var built = false;
 
     var types = ['less', 'drive', 'migrate', 'sf', 'team', 'pad', 'end'];
-    var current;
+    var current, progress;
     var makeList = function (data) {
         var c = types.indexOf(data.type);
         current = c;
@@ -307,7 +307,7 @@ button.primary:hover{
         };
         var list = '<ul>';
         types.forEach(function (el, i) {
-            if (i >= 6) { return; }
+            if (el === "end") { return; }
             list += getLi(i);
         });
         list += '</ul>';
@@ -315,7 +315,7 @@ button.primary:hover{
     };
     var makeBar = function (data) {
         var c = types.indexOf(data.type);
-        var l = types.length;
+        var l = types.length - 1; // don't count "end" as a type
         var progress = Math.min(data.progress, 100);
         var p = (progress / l) + (100 * c / l);
         var bar = '<div class="cp-loading-progress-bar">'+
@@ -327,20 +327,34 @@ button.primary:hover{
     var hasErrored = false;
     var updateLoadingProgress = function (data) {
         if (!built || !data) { return; }
+
+        // Make sure progress doesn't go backward
         var c = types.indexOf(data.type);
-        if (c < current) { return console.error(data); }
+        if (c < current) { return console.debug(data); }
+        if (c === current && progress > data.progress) { return console.debug(data); }
+        progress = data.progress;
+
         try {
-            document.querySelector('.cp-loading-spinner-container').style.display = 'none';
+            var el1 = document.querySelector('.cp-loading-spinner-container');
-            document.querySelector('.cp-loading-progress-list').innerHTML = makeList(data);
+            if (el1) { el1.style.display = 'none'; }
-            document.querySelector('.cp-loading-progress-container').innerHTML = makeBar(data);
+            var el2 = document.querySelector('.cp-loading-progress-list');
+            if (el2) { el2.innerHTML = makeList(data); }
+            var el3 = document.querySelector('.cp-loading-progress-container');
+            if (el3) { el3.innerHTML = makeBar(data); }
         } catch (e) {
-            if (!hasErrored) { console.error(e); }
+            //if (!hasErrored) { console.error(e); }
         }
     };
     window.CryptPad_updateLoadingProgress = updateLoadingProgress;
 
     window.CryptPad_loadingError = function (err) {
         if (!built) { return; }
+
+        if (err === 'Error: XDR encoding failure') {
+            console.warn(err);
+            return;
+        }
+
         hasErrored = true;
         var err2;
         if (err === 'Script error.') {

customize.dist/pages.js

@@ -69,7 +69,7 @@ define([
     Msg.footer_team = "Contributors"; // XXX existing key
     Msg.footer_tos = "Terms of Service"; // XXX existing key
 
-    Pages.versionString = "v3.24.0 (YunnanLakeNewt)";
+    Pages.versionString = "v3.25.0 (ZyzomysPedunculatus)";
 
     // used for the about menu
     Pages.imprintLink = AppConfig.imprint ? footLink(imprintUrl, 'imprint') : undefined;

customize.dist/src/less2/include/fileupload.less

@@ -14,7 +14,7 @@
         right: 10vw;
         bottom: 10vh;
         box-sizing: border-box;
-        z-index: 100000; //Z file upload table container
+        z-index: 100001; //Z file upload table container: just above the file picker
         display: none;
         color: darken(@colortheme_static_apps[default], 10%);
         max-height: 180px;

customize.dist/src/less2/include/forms.less

@@ -2,6 +2,10 @@
 @import (reference) "./variables.less";
 
 .forms_main() {
+    --LessLoader_require: LessLoader_currentFile();
+}
+
+& {
     @alertify-fore: @colortheme_modal-fg;
     @alertify-btn-fg: @alertify-fore;
     @alertify-light-bg: fade(@alertify-fore, 25%);
@@ -114,7 +118,9 @@
             margin: 0;
         }
 
-        &:hover, &:active, &:focus {
+        &:hover, &:not(:disabled):not(.disabled):active, &:focus {
+            color: @alertify-btn-fg;
+            border: 1px solid @alertify-btn-fg;
             background-color: lighten(@alertify-fore, 35%);
         }
 
@@ -124,19 +130,32 @@
             font-weight: bold;
         }
 
+        &.btn-default {
+            border-color: @cryptpad_text_col;
+            color: @cryptpad_text_col;
+            &:hover, &:not(:disabled):active, &:focus {
+                border-color: @cryptpad_text_col;
+                color: @cryptpad_text_col;
+                background-color: #ccc;
+            }
+        }
+
         &.danger, &.btn-danger {
             background-color: @colortheme_alertify-red;
             border-color: @colortheme_alertify-red-border;
             color: @colortheme_alertify-red-color;
-            &:hover, &:active, &:focus {
+            &:hover, &:not(:disabled):active, &:focus {
+                border-color: @colortheme_alertify-red-border;
+                color: @colortheme_alertify-red-color;
                 background-color: contrast(@colortheme_modal-bg, darken(@colortheme_alertify-red, 10%), lighten(@colortheme_alertify-red, 10%));
             }
         }
 
-        &.danger-alt, &.btn-danger-alt {
+        &.danger-alt, &.btn-danger-alt, &.btn-danger-outline {
             border-color: @colortheme_alertify-red;
             color: @colortheme_alertify-red;
-            &:hover, &:active, &:focus {
+            &:hover, &:not(:disabled):active, &:focus {
+                border-color: @colortheme_alertify-red;
                 color: @colortheme_alertify-red-color;
                 background-color: contrast(@colortheme_modal-bg, darken(@colortheme_alertify-red, 10%), lighten(@colortheme_alertify-red, 10%));
             }
@@ -146,17 +165,21 @@
             background-color: @colortheme_alertify-green;
             border-color: @colortheme_alertify-green-border;
             color: @colortheme_alertify-green-color;
-            &:hover, &:active, &:focus {
+            &:hover, &:not(:disabled):active, &:focus {
+                border-color: @colortheme_alertify-green-border;
+                color: @colortheme_alertify-green-color;
                 background-color: contrast(@colortheme_modal-bg, darken(@colortheme_alertify-green, 10%), lighten(@colortheme_alertify-green, 10%));
             }
         }
 
-        &.primary, &.btn-primary {
+        &.primary, &.btn-primary, &.btn-success {
             background-color: @colortheme_alertify-primary;
             color: @colortheme_alertify-primary-text;
             border-color: @colortheme_alertify-primary-border;
             font-weight: bold;
-            &:hover, &:active, &:focus {
+            &:hover, &:not(:disabled):active, &:focus {
+                color: @colortheme_alertify-primary-text;
+                border-color: @colortheme_alertify-primary-border;
                 background-color: contrast(@colortheme_modal-bg, darken(@colortheme_alertify-primary, 10%), lighten(@colortheme_alertify-primary, 10%));
             }
         }
@@ -165,7 +188,9 @@
             border-color: @cryptpad_text_col;
             color: @cryptpad_text_col;
             background-color: transparent;
-            &:hover, &:hover, &:focus {
+            &:hover, &:not(:disabled):active, &:focus {
+                border-color: @cryptpad_text_col;
+                color: @cryptpad_text_col;
                 background-color: fade(@cryptpad_text_col, 25%);
             }
         }
@@ -173,7 +198,9 @@
         &.cancel, &.btn-cancel {
             border-color: @colortheme_alertify-cancel-border;
             color: @colortheme_alertify-cancel-border;
-            &:hover, &:hover, &:focus {
+            &:hover, &:not(:disabled):active, &:focus {
+                border-color: @colortheme_alertify-cancel-border;
+                color: @colortheme_alertify-cancel-border;
                 background-color: fade(@colortheme_alertify-cancel-border, 25%);
             }
         }
@@ -185,7 +212,7 @@
 
         &:focus {
             //border: 1px dotted @alertify-base;
-            box-shadow: 0px 0px 5px @colortheme_alertify-primary;
+            box-shadow: 0px 0px 5px @colortheme_alertify-primary !important;
             outline: none;
         }
         &::-moz-focus-inner {
@@ -202,5 +229,4 @@
             }
         }
     }
-
 }

customize.dist/src/less2/include/markdown.less

@@ -64,6 +64,57 @@
     }
 }
 
+.mediatag_cryptpad() {
+    media-tag {
+        &:empty {
+            display: none !important;
+        }
+        cursor: pointer;
+        * {
+            max-width: 100%;
+        }
+        iframe[src$=".pdf"] {
+            width: 100%;
+            height: 80vh;
+            max-height: 90vh;
+        }
+        button.mediatag-download-btn {
+            flex-flow: column;
+            & > span {
+                display: flex;
+                line-height: 1.5;
+                align-items: center;
+                &.mediatag-download-name b {
+                    white-space: nowrap;
+                    overflow: hidden;
+                    text-overflow: ellipsis;
+                }
+            }
+        }
+        button.btn-default {
+            display: inline-flex;
+            max-width: 250px;
+            min-height: 38px;
+            justify-content: center;
+            .fa {
+                margin-right: 5px;
+            }
+            b {
+                margin-left: 5px;
+                white-space: nowrap;
+                overflow: hidden;
+                text-overflow: ellipsis;
+            }
+        }
+    }
+    media-tag:empty {
+        width: 100px;
+        height: 100px;
+        display: inline-block;
+        border: 1px solid #BBB;
+    }
+}
+
 .markdown_cryptpad() {
     word-wrap: break-word;
 
@@ -84,23 +135,8 @@
             margin-top: 4px;
         }
     }
-    media-tag {
+
-        cursor: pointer;
+    .mediatag_cryptpad();
-        * {
-            max-width: 100%;
-        }
-        iframe[src$=".pdf"] {
-            width: 100%;
-            height: 80vh;
-            max-height: 90vh;
-        }
-    }
-    media-tag:empty {
-        width: 100px;
-        height: 100px;
-        display: inline-block;
-        border: 1px solid #BBB;
-    }
 
     pre.markmap {
         border: 1px solid #ddd;

customize.dist/src/less2/include/modals-ui-elements.less

@@ -1,6 +1,7 @@
 @import (reference) "./colortheme-all.less";
 @import (reference) "./variables.less";
 @import (reference) "./browser.less";
+@import (reference) "./markdown.less";
 
 .modals-ui-elements_main() {
     --LessLoader_require: LessLoader_currentFile();
@@ -214,6 +215,7 @@
                 flex: 1;
                 min-width: 0;
                 overflow: auto;
+                .mediatag_cryptpad();
                 media-tag {
                     & > * {
                         max-width: 100%;

customize.dist/src/less2/include/sidebar-layout.less

@@ -118,7 +118,7 @@
                     //border-radius: 0 0.25em 0.25em 0;
                     //border: 1px solid #adadad;
                     border-left: 0px;
-                    height: @variables_input-height;
+                    height: 40px;
                     margin: 0 !important;
                 }
             }

customize.dist/src/less2/include/support.less

@@ -78,7 +78,7 @@
             }
             &.cp-support-list-closed {
                 .cp-support-list-actions {
-                    display: block !important;
+                    display: flex !important;
                     .cp-support-answer, .cp-support-close {
                         display: none;
                     }

customize.dist/src/less2/pages/page-login.less

@@ -2,10 +2,12 @@
 @import (reference) "../include/colortheme-all.less";
 @import (reference) "../include/alertify.less";
 @import (reference) "../include/checkmark.less";
+@import (reference) "../include/forms.less";
 
 &.cp-page-login {
     .infopages_main();
     .alertify_main();
+    .forms_main();
     .checkmark_main(20px);
 
     .form-group {

docs/cryptpad.service

@@ -17,7 +17,7 @@ SyslogIdentifier=cryptpad
 User=cryptpad
 Group=cryptpad
 # modify to match your working directory
-Environment='PWD="/home/cryptpad/cryptpad/cryptpad"'
+Environment='PWD="/home/cryptpad/cryptpad"'
 
 # systemd sets the open file limit to 4000 unless you override it
 # cryptpad stores its data with the filesystem, so you should increase this to match the value of `ulimit -n`

docs/example.nginx.conf

@@ -32,6 +32,9 @@ server {
     server_name your-main-domain.com your-sandbox-domain.com;
 
     # You'll need to Set the path to your certificates and keys here
+    # IMPORTANT: this config is intended to serve assets for at least two domains
+    # (your main domain and your sandbox domain). As such, you'll need to generate a single SSL certificate
+    # that includes both domains in order for things to work as expected.
     ssl_certificate         /home/cryptpad/.acme.sh/your-main-domain.com/fullchain.cer;
     ssl_certificate_key     /home/cryptpad/.acme.sh/your-main-domain.com/your-main-domain.com.key;
     ssl_trusted_certificate /home/cryptpad/.acme.sh/your-main-domain.com/ca.cer;
@@ -177,8 +180,8 @@ server {
         add_header Cache-Control max-age=31536000;
         add_header 'Access-Control-Allow-Origin' '*';
         add_header 'Access-Control-Allow-Methods' 'GET, POST, OPTIONS';
-        add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range';
+        add_header 'Access-Control-Allow-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range,Content-Length';
-        add_header 'Access-Control-Expose-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range';
+        add_header 'Access-Control-Expose-Headers' 'DNT,X-CustomHeader,Keep-Alive,User-Agent,X-Requested-With,If-Modified-Since,Cache-Control,Content-Type,Content-Range,Range,Content-Length';
         try_files $uri =404;
     }
 

lib/commands/admin-rpc.js

@@ -56,6 +56,11 @@ var getCacheStats = function (env, server, cb) {
     });
 };
 
+// CryptPad_AsyncStore.rpc.send('ADMIN', ['GET_WORKER_PROFILES'], console.log)
+var getWorkerProfiles = function (Env, Server, cb) {
+    cb(void 0, Env.commandTimers);
+};
+
 var getActiveSessions = function (Env, Server, cb) {
     var stats = Server.getSessionStats();
     cb(void 0, [
@@ -155,9 +160,19 @@ var archiveDocument = function (Env, Server, cb, data) {
     switch (id.length) {
         case 32:
         // TODO disconnect users from active sessions
-            return void Env.msgStore.archiveChannel(id, cb);
+            return void Env.msgStore.archiveChannel(id, Util.both(cb, function (err) {
+                Env.Log.info("ARCHIVAL_CHANNEL_BY_ADMIN_RPC", {
+                    channelId: id,
+                    status: err? String(err): "SUCCESS",
+                });
+            }));
         case 48:
-            return void Env.blobStore.archive.blob(id, cb);
+            return void Env.blobStore.archive.blob(id, Util.both(cb, function (err) {
+                Env.Log.info("ARCHIVAL_BLOB_BY_ADMIN_RPC", {
+                    id: id,
+                    status: err? String(err): "SUCCESS",
+                });
+            }));
         default:
             return void cb("INVALID_ID_LENGTH");
     }
@@ -167,12 +182,30 @@ var archiveDocument = function (Env, Server, cb, data) {
     // Env.blobStore.archive.proof(userSafeKey, blobId, cb)
 };
 
-var restoreArchivedDocument = function (Env, Server, cb) {
+var restoreArchivedDocument = function (Env, Server, cb, data) {
-    // Env.msgStore.restoreArchivedChannel(channelName, cb)
+    var id = Array.isArray(data) && data[1];
-    // Env.blobStore.restore.blob(blobId, cb)
+    if (typeof(id) !== 'string' || id.length < 32) { return void cb("EINVAL"); }
-    // Env.blobStore.restore.proof(userSafekey, blobId, cb)
 
-    cb("NOT_IMPLEMENTED");
+    switch (id.length) {
+        case 32:
+            return void Env.msgStore.restoreArchivedChannel(id, Util.both(cb, function (err) {
+                Env.Log.info("RESTORATION_CHANNEL_BY_ADMIN_RPC", {
+                    id: id,
+                    status: err? String(err): 'SUCCESS',
+                });
+            }));
+        case 48:
+            // FIXME this does not yet restore blob ownership
+            // Env.blobStore.restore.proof(userSafekey, id, cb)
+            return void Env.blobStore.restore.blob(id, Util.both(cb, function (err) {
+                Env.Log.info("RESTORATION_BLOB_BY_ADMIN_RPC", {
+                    id: id,
+                    status: err? String(err): 'SUCCESS',
+                });
+            }));
+        default:
+            return void cb("INVALID_ID_LENGTH");
+    }
 };
 
 // CryptPad_AsyncStore.rpc.send('ADMIN', ['CLEAR_CACHED_CHANNEL_INDEX', documentID], console.log)
@@ -315,6 +348,7 @@ var commands = {
     INSTANCE_STATUS: instanceStatus,
     GET_LIMITS: getLimits,
     SET_LAST_EVICTION: setLastEviction,
+    GET_WORKER_PROFILES: getWorkerProfiles,
 };
 
 Admin.command = function (Env, safeKey, data, _cb, Server) {

lib/commands/metadata.js

@@ -13,11 +13,19 @@ Data.getMetadataRaw = function (Env, channel /* channelName */, _cb) {
 
     var cached = Env.metadata_cache[channel];
     if (HK.isMetadataMessage(cached)) {
+        Env.checkCache(channel);
         return void cb(void 0, cached);
     }
 
     Env.batchMetadata(channel, cb, function (done) {
-        Env.computeMetadata(channel, done);
+        Env.computeMetadata(channel, function (err, meta) {
+            if (!err && HK.isMetadataMessage(meta)) {
+                Env.metadata_cache[channel] = meta;
+                // clear metadata after a delay if nobody has joined the channel within 30s
+                Env.checkCache(channel);
+            }
+            done(err, meta);
+        });
     });
 };
 

lib/commands/upload.js

@@ -75,21 +75,9 @@ Upload.upload = function (Env, safeKey, chunk, cb) {
     Env.blobStore.upload(safeKey, chunk, cb);
 };
 
-var reportStatus = function (Env, label, safeKey, err, id) {
-    var data = {
-        safeKey: safeKey,
-        err: err && err.message || err,
-        id: id,
-    };
-    var method = err? 'error': 'info';
-    Env.Log[method](label, data);
-};
-
 Upload.complete = function (Env, safeKey, arg, cb) {
-    Env.blobStore.complete(safeKey, arg, function (err, id) {
+    Env.blobStore.closeBlobstage(safeKey);
-        reportStatus(Env, 'UPLOAD_COMPLETE', safeKey, err, id);
+    Env.completeUpload(safeKey, arg, false, cb);
-        cb(err, id);
-    });
 };
 
 Upload.cancel = function (Env, safeKey, arg, cb) {
@@ -97,9 +85,9 @@ Upload.cancel = function (Env, safeKey, arg, cb) {
 };
 
 Upload.complete_owned = function (Env, safeKey, arg, cb) {
-    Env.blobStore.completeOwned(safeKey, arg, function (err, id) {
+    Env.blobStore.closeBlobstage(safeKey);
-        reportStatus(Env, 'UPLOAD_COMPLETE_OWNED', safeKey, err, id);
+    var user = Core.getSession(Env.Sessions, safeKey);
-        cb(err, id);
+    var size = user.pendingUploadSize;
-    });
+    Env.completeUpload(safeKey, arg, true, size, cb);
 };
 

lib/env.js

@@ -42,6 +42,8 @@ module.exports.create = function (config) {
 
         metadata_cache: {},
         channel_cache: {},
+        cache_checks: {},
+
         queueStorage: WriteQueue(),
         queueDeletes: WriteQueue(),
         queueValidation: WriteQueue(),
@@ -94,6 +96,7 @@ module.exports.create = function (config) {
         disableIntegratedEviction: config.disableIntegratedEviction || false,
         lastEviction: +new Date(),
         evictionReport: {},
+        commandTimers: {},
     };
 
     (function () {
@@ -116,8 +119,14 @@ module.exports.create = function (config) {
         }
     }());
 
-
+    Env.checkCache = function (channel) {
-
+        var f = Env.cache_checks[channel] || Util.throttle(function () {
+            delete Env.cache_checks[channel];
+            if (Env.channel_cache[channel]) { return; }
+            delete Env.metadata_cache[channel];
+        }, 30000);
+        f();
+    };
 
     (function () {
         var custom = config.customLimits;

lib/hk-util.js

@@ -419,9 +419,11 @@ const getHistoryOffset = (Env, channelName, lastKnownHash, _cb) => {
             // fall through to the next block if the offset of the hash in question is not in memory
             if (lastKnownHash && typeof(lkh) !== "number") { return; }
 
+            // If we have a lastKnownHash or we didn't ask for one, we don't need the next blocks
+            waitFor.abort();
+
             // Since last 2 checkpoints
             if (!lastKnownHash) {
-                waitFor.abort();
                 // Less than 2 checkpoints in the history: return everything
                 if (index.cpIndex.length < 2) { return void cb(null, 0); }
                 // Otherwise return the second last checkpoint's index
@@ -436,7 +438,15 @@ const getHistoryOffset = (Env, channelName, lastKnownHash, _cb) => {
                     to reconcile their differences. */
             }
 
-            offset = lkh;
+            // If our lastKnownHash is older than the 2nd to last checkpoint, send
+            // EUNKNOWN to tell the user to empty their cache
+            if (lkh && index.cpIndex.length >= 2 && lkh < index.cpIndex[0].offset) {
+                waitFor.abort();
+                return void cb(new Error('EUNKNOWN'));
+            }
+
+            // Otherwise use our lastKnownHash
+            cb(null, lkh);
         }));
     }).nThen((w) => {
         // skip past this block if the offset is anything other than -1

lib/storage/blob.js

@@ -139,6 +139,15 @@ var upload = function (Env, safeKey, content, cb) {
     }
 };
 
+var closeBlobstage = function (Env, safeKey) {
+    var session = Env.getSession(safeKey);
+    if (!(session && session.blobstage && typeof(session.blobstage.close) === 'function')) {
+        return;
+    }
+    session.blobstage.close();
+    delete session.blobstage;
+};
+
 // upload_cancel
 var upload_cancel = function (Env, safeKey, fileSize, cb) {
     var session = Env.getSession(safeKey);
@@ -159,27 +168,22 @@ var upload_cancel = function (Env, safeKey, fileSize, cb) {
 
 // upload_complete
 var upload_complete = function (Env, safeKey, id, cb) {
-    var session = Env.getSession(safeKey);
+    closeBlobstage(Env, safeKey);
-
-    if (session.blobstage && session.blobstage.close) {
-        session.blobstage.close();
-        delete session.blobstage;
-    }
 
     var oldPath = makeStagePath(Env, safeKey);
     var newPath = makeBlobPath(Env, id);
 
     nThen(function (w) {
         // make sure the path to your final location exists
-        Fse.mkdirp(Path.dirname(newPath), function (e) {
+        Fse.mkdirp(Path.dirname(newPath), w(function (e) {
             if (e) {
                 w.abort();
                 return void cb('RENAME_ERR');
             }
-        });
+        }));
     }).nThen(function (w) {
         // make sure there's not already something in that exact location
-        isFile(newPath, function (e, yes) {
+        isFile(newPath, w(function (e, yes) {
             if (e) {
                 w.abort();
                 return void cb(e);
@@ -188,8 +192,8 @@ var upload_complete = function (Env, safeKey, id, cb) {
                 w.abort();
                 return void cb('RENAME_ERR');
             }
-            cb(void 0, newPath, id);
+            cb(void 0, id);
-        });
+        }));
     }).nThen(function () {
         // finally, move the old file to the new path
         // FIXME we could just move and handle the EEXISTS instead of the above block
@@ -217,15 +221,7 @@ var tryId = function (path, cb) {
 
 // owned_upload_complete
 var owned_upload_complete = function (Env, safeKey, id, cb) {
-    var session = Env.getSession(safeKey);
+    closeBlobstage(Env, safeKey);
-
-    // the file has already been uploaded to the staging area
-    // close the pending writestream
-    if (session.blobstage && session.blobstage.close) {
-        session.blobstage.close();
-        delete session.blobstage;
-    }
-
     if (!isValidId(id)) {
         return void cb('EINVAL_ID');
     }
@@ -582,6 +578,9 @@ BlobStore.create = function (config, _cb) {
                 },
             },
 
+            closeBlobstage: function (safeKey) {
+                closeBlobstage(Env, safeKey);
+            },
             complete: function (safeKey, id, _cb) {
                 var cb = Util.once(Util.mkAsync(_cb));
                 if (!isValidSafeKey(safeKey)) { return void cb('INVALID_SAFEKEY'); }

lib/storage/file.js

@@ -1,6 +1,6 @@
 /*@flow*/
 /* jshint esversion: 6 */
-/* global Buffer */
+/* globals Buffer */
 var Fs = require("fs");
 var Fse = require("fs-extra");
 var Path = require("path");
@@ -66,6 +66,10 @@ var mkTempPath = function (env, channelId) {
     return mkPath(env, channelId) + '.temp';
 };
 
+var mkOffsetPath = function (env, channelId) {
+    return mkPath(env, channelId) + '.offset';
+};
+
 // pass in the path so we can reuse the same function for archived files
 var channelExists = function (filepath, cb) {
     Fs.stat(filepath, function (err, stat) {
@@ -131,7 +135,9 @@ const readMessagesBin = (env, id, start, msgHandler, cb) => {
     const collector = createIdleStreamCollector(stream);
     const handleMessageAndKeepStreamAlive = Util.both(msgHandler, collector.keepAlive);
     const done = Util.both(cb, collector);
-    return void readFileBin(stream, handleMessageAndKeepStreamAlive, done);
+    return void readFileBin(stream, handleMessageAndKeepStreamAlive, done, {
+        offset: start,
+    });
 };
 
 // reads classic metadata from a channel log and aborts
@@ -190,6 +196,37 @@ var closeChannel = function (env, channelName, cb) {
     }
 };
 
+var clearOffset = function (env, channelId, cb) {
+    var path = mkOffsetPath(env, channelId);
+    // we should always be able to recover from invalid offsets, so failure to delete them
+    // is not catastrophic. Anything calling this function can optionally ignore errors it might report
+    Fs.unlink(path, cb);
+};
+
+var writeOffset = function (env, channelId, data, cb) {
+    var path = mkOffsetPath(env, channelId);
+    var s_data;
+    try {
+        s_data = JSON.stringify(data);
+    } catch (err) {
+        return void cb(err);
+    }
+    Fs.writeFile(path, s_data, cb);
+};
+
+var getOffset = function (env, channelId, cb) {
+    var path = mkOffsetPath(env, channelId);
+    Fs.readFile(path, function (err, content) {
+        if (err) { return void cb(err); }
+        try {
+            var json = JSON.parse(content);
+            cb(void 0, json);
+        } catch (err2) {
+            cb(err2);
+        }
+    });
+};
+
 // truncates a file to the end of its metadata line
 // TODO write the metadata in a dedicated file
 var clearChannel = function (env, channelId, _cb) {
@@ -213,6 +250,7 @@ var clearChannel = function (env, channelId, _cb) {
                 cb();
             });
         });
+        clearOffset(env, channelId, function () {});
     });
 };
 
@@ -389,6 +427,7 @@ var removeChannel = function (env, channelName, cb) {
                 CB(labelError("E_METADATA_REMOVAL", err));
             }
         }));
+        clearOffset(env, channelName, w());
     }).nThen(function () {
         if (errors === 2) {
             return void CB(labelError('E_REMOVE_CHANNEL', new Error("ENOENT")));
@@ -604,6 +643,8 @@ var archiveChannel = function (env, channelName, cb) {
                 return void cb(err);
             }
         }));
+    }).nThen(function (w) {
+        clearOffset(env, channelName, w());
     }).nThen(function (w) {
         // archive the dedicated metadata channel
         var metadataPath = mkMetadataPath(env, channelName);
@@ -861,6 +902,7 @@ var trimChannel = function (env, channelName, hash, _cb) {
             }
         }));
     }).nThen(function (w) {
+        clearOffset(env, channelName, w());
         cleanUp(w(function (err) {
             if (err) {
                 w.abort();
@@ -1177,6 +1219,25 @@ module.exports.create = function (conf, _cb) {
                 });
             },
 
+        // OFFSETS
+// these exist strictly as an optimization
+// you can always remove them without data loss
+            clearOffset: function (channelName, _cb) {
+                var cb = Util.once(Util.mkAsync(_cb));
+                if (!isValidChannelId(channelName)) { return void cb(new Error('EINVAL')); }
+                clearOffset(env, channelName, cb);
+            },
+            writeOffset: function (channelName, data, _cb) {
+                var cb = Util.once(Util.mkAsync(_cb));
+                if (!isValidChannelId(channelName)) { return void cb(new Error('EINVAL')); }
+                writeOffset(env, channelName, data, cb);
+            },
+            getOffset: function (channelName, _cb) {
+                var cb = Util.once(Util.mkAsync(_cb));
+                if (!isValidChannelId(channelName)) { return void cb(new Error('EINVAL')); }
+                getOffset(env, channelName, cb);
+            },
+
         // METADATA METHODS
             // fetch the metadata for a channel
             getChannelMetadata: function (channelName, cb) {

lib/stream-file.js

@@ -44,8 +44,8 @@ const mkBufferSplit = () => {
 
 // return a streaming function which transforms buffers into objects
 // containing the buffer and the offset from the start of the stream
-const mkOffsetCounter = () => {
+const mkOffsetCounter = (offset) => {
-    let offset = 0;
+    offset = offset || 0;
     return Pull.map((buff) => {
         const out = { offset: offset, buff: buff };
         // +1 for the eaten newline
@@ -59,13 +59,14 @@ const mkOffsetCounter = () => {
 // that this function has a lower memory profile than our classic method
 // of reading logs line by line.
 // it also allows the handler to abort reading at any time
-Stream.readFileBin = (stream, msgHandler, cb) => {
+Stream.readFileBin = (stream, msgHandler, cb, opt) => {
+    opt = opt || {};
     //const stream = Fs.createReadStream(path, { start: start });
     let keepReading = true;
     Pull(
         ToPull.read(stream),
         mkBufferSplit(),
-        mkOffsetCounter(),
+        mkOffsetCounter(opt.offset),
         Pull.asyncMap((data, moreCb) => {
             msgHandler(data, moreCb, () => {
                 try {

lib/workers/db-worker.js

@@ -1,5 +1,5 @@
 /* jshint esversion: 6 */
-/* global process */
+/* globals process, Buffer */
 
 const HK = require("../hk-util");
 const Store = require("../storage/file");
@@ -30,6 +30,11 @@ Logger.levels.forEach(function (level) {
     };
 });
 
+var DETAIL = 1000;
+var round = function (n) {
+    return Math.floor(n * DETAIL) / DETAIL;
+};
+
 var ready = false;
 var store;
 var pinStore;
@@ -114,14 +119,15 @@ const init = function (config, _cb) {
             * including the initial metadata line, if it exists
 
 */
-const computeIndex = function (data, cb) {
-    if (!data || !data.channel) {
-        return void cb('E_NO_CHANNEL');
-    }
 
-    const channelName = data.channel;
+const OPEN_CURLY_BRACE = Buffer.from('{');
+const CHECKPOINT_PREFIX = Buffer.from('cp|');
+const isValidOffsetNumber = function (n) {
+    return typeof(n) === 'number' && n >= 0;
+};
 
-    const cpIndex = [];
+const computeIndexFromOffset = function (channelName, offset, cb) {
+    let cpIndex = [];
     let messageBuf = [];
     let i = 0;
 
@@ -129,27 +135,42 @@ const computeIndex = function (data, cb) {
 
     const offsetByHash = {};
     let offsetCount = 0;
-    let size = 0;
+    let size = offset || 0;
+    var start = offset || 0;
+    let unconventional = false;
+
     nThen(function (w) {
         // iterate over all messages in the channel log
         // old channels can contain metadata as the first message of the log
         // skip over metadata as that is handled elsewhere
         // otherwise index important messages in the log
-        store.readMessagesBin(channelName, 0, (msgObj, readMore) => {
+        store.readMessagesBin(channelName, start, (msgObj, readMore, abort) => {
             let msg;
             // keep an eye out for the metadata line if you haven't already seen it
             // but only check for metadata on the first line
-            if (!i && msgObj.buff.indexOf('{') === 0) {
+            if (i) {
-                i++; // always increment the message counter
+                // fall through intentionally because the following blocks are invalid
+                // for all but the first message
+            } else if (msgObj.buff.includes(OPEN_CURLY_BRACE)) {
                 msg = HK.tryParse(Env, msgObj.buff.toString('utf8'));
-                if (typeof msg === "undefined") { return readMore(); }
+                if (typeof msg === "undefined") {
+                    i++; // always increment the message counter
+                    return readMore();
+                }
 
                 // validate that the current line really is metadata before storing it as such
                 // skip this, as you already have metadata...
-                if (HK.isMetadataMessage(msg)) { return readMore(); }
+                if (HK.isMetadataMessage(msg)) {
+                    i++; // always increment the message counter
+                    return readMore();
+                }
+            } else if (!(msg = HK.tryParse(Env, msgObj.buff.toString('utf8')))) {
+                w.abort();
+                abort();
+                return CB("OFFSET_ERROR");
             }
             i++;
-            if (msgObj.buff.indexOf('cp|') > -1) {
+            if (msgObj.buff.includes(CHECKPOINT_PREFIX)) {
                 msg = msg || HK.tryParse(Env, msgObj.buff.toString('utf8'));
                 if (typeof msg === "undefined") { return readMore(); }
                 // cache the offsets of checkpoints if they can be parsed
@@ -164,6 +185,7 @@ const computeIndex = function (data, cb) {
                 }
             } else if (messageBuf.length > 100 && cpIndex.length === 0) {
                 // take the last 50 messages
+                unconventional = true;
                 messageBuf = messageBuf.slice(-50);
             }
             // if it's not metadata or a checkpoint then it should be a regular message
@@ -192,11 +214,40 @@ const computeIndex = function (data, cb) {
                 size = msgObj.offset + msgObj.buff.length + 1;
             });
         }));
+    }).nThen(function (w) {
+        cpIndex = HK.sliceCpIndex(cpIndex, i);
+
+        var new_start;
+        if (cpIndex.length) {
+            new_start = cpIndex[0].offset;
+        } else if (unconventional && messageBuf.length && isValidOffsetNumber(messageBuf[0].offset)) {
+            new_start = messageBuf[0].offset;
+        }
+
+        if (new_start === start) { return; }
+        if (!isValidOffsetNumber(new_start)) { return; }
+
+        // store the offset of the earliest relevant line so that you can start from there next time...
+        store.writeOffset(channelName, {
+            start: new_start,
+            created: +new Date(),
+        }, w(function () {
+            var diff = new_start - start;
+            Env.Log.info('WORKER_OFFSET_UPDATE', {
+                channel: channelName,
+                start: start,
+                startMB: round(start / 1024 / 1024),
+                update: new_start,
+                updateMB: round(new_start / 1024 / 1024),
+                diff: diff,
+                diffMB: round(diff / 1024 / 1024),
+            });
+        }));
     }).nThen(function () {
         // return the computed index
         CB(null, {
             // Only keep the checkpoints included in the last 100 messages
-            cpIndex: HK.sliceCpIndex(cpIndex, i),
+            cpIndex: cpIndex,
             offsetByHash: offsetByHash,
             offsets: offsetCount,
             size: size,
@@ -206,6 +257,47 @@ const computeIndex = function (data, cb) {
     });
 };
 
+const computeIndex = function (data, cb) {
+    if (!data || !data.channel) {
+        return void cb('E_NO_CHANNEL');
+    }
+
+    const channelName = data.channel;
+    const CB = Util.once(cb);
+
+    var start = 0;
+    nThen(function (w) {
+        store.getOffset(channelName, w(function (err, obj) {
+            if (err) { return; }
+            if (obj && typeof(obj.start) === 'number' && obj.start > 0) {
+                start = obj.start;
+                Env.Log.verbose('WORKER_OFFSET_RECOVERY', {
+                    channel: channelName,
+                    start: start,
+                    startMB: round(start / 1024 / 1024),
+                });
+            }
+        }));
+    }).nThen(function (w) {
+        computeIndexFromOffset(channelName, start, w(function (err, index) {
+            if (err === 'OFFSET_ERROR') {
+                return Env.Log.error("WORKER_OFFSET_ERROR", {
+                    channel: channelName,
+                });
+            }
+            w.abort();
+            CB(err, index);
+        }));
+    }).nThen(function (w) {
+        // if you're here there was an OFFSET_ERROR..
+        // first remove the offset that caused the problem to begin with
+        store.clearOffset(channelName, w());
+    }).nThen(function () {
+        // now get the history as though it were the first time
+        computeIndexFromOffset(channelName, 0, CB);
+    });
+};
+
 const computeMetadata = function (data, cb) {
     const ref = {};
     const lineHandler = Meta.createLineHandler(ref, Env.Log.error);
@@ -457,6 +549,41 @@ const evictInactive = function (data, cb) {
     Eviction(Env, cb);
 };
 
+var reportStatus = function (Env, label, safeKey, err, id, size) {
+    var data = {
+        safeKey: safeKey,
+        err: err && err.message || err,
+        id: id,
+        size: size,
+        sizeMB: round((size || 0) / 1024 / 1024),
+    };
+    var method = err? 'error': 'info';
+    Env.Log[method](label, data);
+};
+
+const completeUpload = function (data, cb) {
+    if (!data) { return void cb('INVALID_ARGS'); }
+    var owned = data.owned;
+    var safeKey = data.safeKey;
+    var arg = data.arg;
+    var size = data.size;
+
+    var method;
+    var label;
+    if (owned) {
+        method = 'completeOwned';
+        label = 'UPLOAD_COMPLETE_OWNED';
+    } else {
+        method = 'complete';
+        label = 'UPLOAD_COMPLETE';
+    }
+
+    Env.blobStore[method](safeKey, arg, function (err, id) {
+        reportStatus(Env, label, safeKey, err, id, size);
+        cb(err, id);
+    });
+};
+
 const COMMANDS = {
     COMPUTE_INDEX: computeIndex,
     COMPUTE_METADATA: computeMetadata,
@@ -471,6 +598,7 @@ const COMMANDS = {
     RUN_TASKS: runTasks,
     WRITE_TASK: writeTask,
     EVICT_INACTIVE: evictInactive,
+    COMPLETE_UPLOAD: completeUpload,
 };
 
 COMMANDS.INLINE = function (data, cb) {
@@ -568,7 +696,7 @@ process.on('message', function (data) {
 
     const cb = function (err, value) {
         process.send({
-            error: err,
+            error: Util.serializeError(err),
             txid: data.txid,
             pid: data.pid,
             value: value,
@@ -577,7 +705,7 @@ process.on('message', function (data) {
 
     if (!ready) {
         return void init(data.config, function (err) {
-            if (err) { return void cb(err); }
+            if (err) { return void cb(Util.serializeError(err)); }
             ready = true;
             cb();
         });

lib/workers/index.js

@@ -9,10 +9,19 @@ const PID = process.pid;
 
 const DB_PATH = 'lib/workers/db-worker';
 const MAX_JOBS = 16;
+const DEFAULT_QUERY_TIMEOUT = 60000 * 15; // increased from three to fifteen minutes because queries for very large files were taking as long as seven minutes
 
 Workers.initialize = function (Env, config, _cb) {
     var cb = Util.once(Util.mkAsync(_cb));
 
+    var incrementTime = function (command, start) {
+        if (!command) { return; }
+        var end = +new Date();
+        var T = Env.commandTimers;
+        var diff = (end - start);
+        T[command] = (T[command] || 0) + (diff / 1000);
+    };
+
     const workers = [];
 
     const response = Util.response(function (errLabel, info) {
@@ -111,8 +120,11 @@ Workers.initialize = function (Env, config, _cb) {
         }
 
         const txid = guid();
+        var start = +new Date();
         var cb = Util.once(Util.mkAsync(Util.both(_cb, function (err /*, value */) {
+            incrementTime(msg && msg.command, start);
             if (err !== 'TIMEOUT') { return; }
+            Log.debug("WORKER_TIMEOUT_CAUSE", msg);
             // in the event of a timeout the user will receive an error
             // but the state used to resend a query in the event of a worker crash
             // won't be cleared. This also leaks a slot that could be used to keep
@@ -132,7 +144,7 @@ Workers.initialize = function (Env, config, _cb) {
         state.tasks[txid] = msg;
 
         // default to timing out affter 180s if no explicit timeout is passed
-        var timeout = typeof(opt.timeout) !== 'undefined'? opt.timeout: 180000;
+        var timeout = typeof(opt.timeout) !== 'undefined'? opt.timeout: DEFAULT_QUERY_TIMEOUT;
         response.expect(txid, cb, timeout);
         state.worker.send(msg);
     };
@@ -422,6 +434,16 @@ Workers.initialize = function (Env, config, _cb) {
             }, cb);
         };
 
+        Env.completeUpload = function (safeKey, arg, owned, size, cb) {
+            sendCommand({
+                command: "COMPLETE_UPLOAD",
+                owned: owned, // Boolean
+                safeKey: safeKey, // String (public key)
+                arg: arg, // String (file id)
+                size: size, // Number || undefined
+            }, cb);
+        };
+
         cb(void 0);
     });
 };

package-lock.json

@@ -1,6 +1,6 @@
 {
   "name": "cryptpad",
-  "version": "3.24.0",
+  "version": "3.25.0",
   "lockfileVersion": 1,
   "requires": true,
   "dependencies": {
@@ -389,16 +389,16 @@
       "optional": true
     },
     "chainpad-crypto": {
-      "version": "0.2.4",
+      "version": "0.2.5",
-      "resolved": "https://registry.npmjs.org/chainpad-crypto/-/chainpad-crypto-0.2.4.tgz",
+      "resolved": "https://registry.npmjs.org/chainpad-crypto/-/chainpad-crypto-0.2.5.tgz",
-      "integrity": "sha512-fWbVyeAv35vf/dkkQaefASlJcEfpEvfRI23Mtn+/TBBry7+LYNuJMXJiovVY35pfyw2+trKh1Py5Asg9vrmaVg==",
+      "integrity": "sha512-K9vRsAspuX+uU1goXPz0CawpLIaOHq+1JP3WfDLqaz67LbCX/MLIUt9aMcSeIJcwZ9uMpqnbMGRktyVPoz6MCA==",
       "requires": {
-        "tweetnacl": "git://github.com/dchest/tweetnacl-js.git#v0.12.2"
+        "tweetnacl": "git+https://github.com/dchest/tweetnacl-js.git#v0.12.2"
       },
       "dependencies": {
         "tweetnacl": {
-          "version": "git://github.com/dchest/tweetnacl-js.git#8a21381d696acdc4e99c9f706f1ad23285795f79",
+          "version": "git+https://github.com/dchest/tweetnacl-js.git#8a21381d696acdc4e99c9f706f1ad23285795f79",
-          "from": "git://github.com/dchest/tweetnacl-js.git#v0.12.2"
+          "from": "git+https://github.com/dchest/tweetnacl-js.git#v0.12.2"
         }
       }
     },

package.json

@@ -1,11 +1,11 @@
 {
   "name": "cryptpad",
   "description": "realtime collaborative visual editor with zero knowlege server",
-  "version": "3.24.0",
+  "version": "3.25.0",
   "license": "AGPL-3.0+",
   "repository": {
     "type": "git",
-    "url": "git://github.com/xwiki-labs/cryptpad.git"
+    "url": "git+https://github.com/xwiki-labs/cryptpad.git"
   },
   "funding": {
     "type": "opencollective",
@@ -13,7 +13,7 @@
   },
   "dependencies": {
     "@mcrowe/minibloom": "^0.2.0",
-    "chainpad-crypto": "^0.2.2",
+    "chainpad-crypto": "^0.2.5",
     "chainpad-server": "^4.0.9",
     "express": "~4.16.0",
     "fs-extra": "^7.0.0",

readme.md

@@ -65,7 +65,7 @@ CryptPad is actively developed by a team at [XWiki SAS](https://www.xwiki.com),
 
 # Contributing
 
-We love Open Source and we love contribution. Learn more about [contributing](https://github.com/xwiki-labs/cryptpad/wiki/Contributor-overview). 
+We love Open Source and we love contribution. Learn more about [contributing](https://docs.cryptpad.fr/en/how_to_contribute.html). 
 
 If you have any questions or comments, or if you're interested in contributing to Cryptpad, come say hi on IRC, `#cryptpad` on Freenode.
 

server.js

@@ -136,6 +136,20 @@ app.head(/^\/common\/feedback\.html/, function (req, res, next) {
 });
 }());
 
+app.use('/blob', function (req, res, next) {
+    if (req.method === 'HEAD') {
+        Express.static(Path.join(__dirname, (config.blobPath || './blob')), {
+            setHeaders: function (res, path, stat) {
+                res.set('Access-Control-Allow-Origin', '*');
+                res.set('Access-Control-Allow-Headers', 'Content-Length');
+                res.set('Access-Control-Expose-Headers', 'Content-Length');
+            }
+        })(req, res, next);
+        return;
+    }
+    next();
+});
+
 app.use(function (req, res, next) {
     if (req.method === 'OPTIONS' && /\/blob\//.test(req.url)) {
         res.setHeader('Access-Control-Allow-Origin', '*');
@@ -202,6 +216,7 @@ var serveConfig = (function () {
                 adminKeys: Env.admins,
                 inactiveTime: Env.inactiveTime,
                 supportMailbox: Env.supportMailbox,
+                defaultStorageLimit: Env.defaultStorageLimit,
                 maxUploadSize: Env.maxUploadSize,
                 premiumUploadSize: Env.premiumUploadSize,
             }, null, '\t'),

www/admin/app-admin.less

@@ -27,6 +27,9 @@
             margin-top: 5px;
         }
     }
+    .cp-admin-setlimit-form + button {
+        margin-top: 5px !important;
+    }
     .cp-admin-getlimits {
         code {
             cursor: pointer;
@@ -58,14 +61,62 @@
 
     .cp-support-container {
         display: flex;
-        flex-flow: column;
+        flex-wrap: wrap;
+        .cp-support-column {
+            min-width: 700px;
+            flex: 1 0 50%;
+            h1 {
+                display: flex;
+                align-items: center;
+                button {
+                    margin-left: 50px !important;
+                }
+            }
+            .cp-support-count {
+                margin-left: 10px;
+            }
+            &.cp-support-column-collapsed {
+                .cp-support-list-ticket {
+                    display: none;
+                }
+            }
+        }
     }
 
     .cp-support-list-actions {
         margin: 10px 0px 10px 2px;
     }
 
+    .cp-support-list-ticket {
+        h2 {
+            font-size: 1.5rem;
+            display: flex;
+            justify-content: space-between;
+            align-items: top;
+            .cp-support-title-buttons {
+                flex-shrink: 0;
+            }
+        }
+        .cp-support-collapsed {
+            display: flex;
+            justify-content: space-between;
+            flex-wrap: wrap;
+            color: #666;
+            .cp-support-ispremium {
+                padding: 0 5px;
+                color: @colortheme_cp-red;
+                background-color: lighten(@colortheme_cp-red, 25%);
+            }
+        }
+    }
+
     .cp-support-list-ticket:not(.cp-support-list-closed) {
+        .cp-support-list-actions {
+            .cp-button-confirm, .cp-support-close {
+                order: 20;
+                margin-left: auto !important;
+            }
+        }
         .cp-support-list-message {
             &:last-child:not(.cp-support-fromadmin)  {
                 color: @colortheme_cp-red;
@@ -85,6 +136,28 @@
             }
         }
     }
+    .cp-support-list-ticket:not(.cp-support-open) {
+        span:first-child {
+            overflow: hidden;
+            text-overflow: ellipsis;
+            white-space: nowrap;
+        }
+        & > :not(h2):not(.cp-support-collapsed) {
+            display: none;
+        }
+        .cp-support-collapse {
+            display: none;
+        }
+        cursor: pointer;
+    }
+    .cp-support-list-ticket.cp-support-open {
+        .cp-support-collapsed {
+            display: none;
+        }
+        .cp-support-expand {
+            display: none;
+        }
+    }
 
     .cp-support-fromadmin {
         color: @colortheme_logo-2;
@@ -93,5 +166,15 @@
             color: @colortheme_logo-2;
         }
     }
+
+    input.cp-admin-inval {
+        border-color: red !important;
+    }
+    .cp-admin-nopassword {
+        .cp-admin-pw {
+            display: none !important;
+        }
+    }
+
 }
 

www/admin/inner.js

@@ -43,6 +43,8 @@ define([
         'general': [
             'cp-admin-flush-cache',
             'cp-admin-update-limit',
+            'cp-admin-archive',
+            'cp-admin-unarchive',
             // 'cp-admin-registration',
         ],
         'quota': [
@@ -107,6 +109,129 @@ define([
         });
         return $div;
     };
+
+    var archiveForm = function (archive, $div, $button) {
+        var label = h('label', { for: 'cp-admin-archive' }, Messages.admin_archiveInput);
+        var input = h('input#cp-admin-archive', {
+            type: 'text'
+        });
+
+        var label2 = h('label.cp-admin-pw', {
+            for: 'cp-admin-archive-pw'
+        }, Messages.admin_archiveInput2);
+        var input2 = UI.passwordInput({
+            id: 'cp-admin-archive-pw',
+            placeholder: Messages.login_password
+        });
+        var $pw = $(input2);
+        $pw.addClass('cp-admin-pw');
+        var $pwInput = $pw.find('input');
+
+
+        $button.before(h('div.cp-admin-setlimit-form', [
+            label,
+            input,
+            label2,
+            input2
+        ]));
+
+        $div.addClass('cp-admin-nopassword');
+
+        var parsed;
+        var $input = $(input).on('keypress change paste', function () {
+            setTimeout(function () {
+                $input.removeClass('cp-admin-inval');
+                var val = $input.val().trim();
+                if (!val) {
+                    $div.toggleClass('cp-admin-nopassword', true);
+                    return;
+                }
+
+                parsed = Hash.isValidHref(val);
+                $pwInput.val('');
+
+                if (!parsed || !parsed.hashData) {
+                    $div.toggleClass('cp-admin-nopassword', true);
+                    return void $input.addClass('cp-admin-inval');
+                }
+
+                var pw = parsed.hashData.version !== 3 && parsed.hashData.password;
+                $div.toggleClass('cp-admin-nopassword', !pw);
+            });
+        });
+        $pw.on('keypress change', function () {
+            setTimeout(function () {
+                $pw.toggleClass('cp-admin-inval', !$pwInput.val());
+            });
+        });
+
+        var clicked = false;
+        $button.click(function () {
+            if (!parsed || !parsed.hashData) {
+                UI.warn(Messages.admin_archiveInval);
+                return;
+            }
+            var pw = parsed.hashData.password ? $pwInput.val() : undefined;
+            var channel;
+            if (parsed.hashData.version === 3) {
+                channel = parsed.hashData.channel;
+            } else {
+                var secret = Hash.getSecrets(parsed.type, parsed.hash, pw);
+                channel = secret && secret.channel;
+            }
+
+            if (!channel) {
+                UI.warn(Messages.admin_archiveInval);
+                return;
+            }
+
+            if (clicked) { return; }
+            clicked = true;
+
+            nThen(function (waitFor) {
+                if (!archive) { return; }
+                common.getFileSize(channel, waitFor(function (err, size) {
+                    if (!err && size === 0) {
+                        clicked = false;
+                        waitFor.abort();
+                        return void UI.warn(Messages.admin_archiveInval);
+                    }
+                }));
+            }).nThen(function () {
+                sFrameChan.query('Q_ADMIN_RPC', {
+                    cmd: archive ? 'ARCHIVE_DOCUMENT' : 'RESTORE_ARCHIVED_DOCUMENT',
+                    data: channel
+                }, function (err, obj) {
+                    var e = err || (obj && obj.error);
+                    clicked = false;
+                    if (e) {
+                        UI.warn(Messages.error);
+                        console.error(e);
+                        return;
+                    }
+                    UI.log(archive ? Messages.archivedFromServer : Messages.restoredFromServer);
+                    $input.val('');
+                    $pwInput.val('');
+                });
+            });
+        });
+    };
+
+    create['archive'] = function () {
+        var key = 'archive';
+        var $div = makeBlock(key, true);
+        var $button = $div.find('button');
+        archiveForm(true, $div, $button);
+        return $div;
+    };
+    create['unarchive'] = function () {
+        var key = 'unarchive';
+        var $div = makeBlock(key, true);
+        var $button = $div.find('button');
+        archiveForm(false, $div, $button);
+        return $div;
+    };
+
     create['registration'] = function () {
         var key = 'registration';
         var $div = makeBlock(key, true);
@@ -222,7 +347,7 @@ define([
                     var keyEl = h('code.cp-limit-key', key);
                     $(keyEl).click(function () {
                         $('.cp-admin-setlimit-form').find('.cp-setlimit-key').val(key);
-                        $('.cp-admin-setlimit-form').find('.cp-setlimit-quota').val(Math.floor(user.limit/1024));
+                        $('.cp-admin-setlimit-form').find('.cp-setlimit-quota').val(Math.floor(user.limit / 1024 / 1024));
                         $('.cp-admin-setlimit-form').find('.cp-setlimit-note').val(user.note);
                     });
                     if (compact) {
@@ -427,7 +552,53 @@ define([
         var $div = $(h('div.cp-support-container')).appendTo($container);
 
         var catContainer = h('div.cp-dropdown-container');
-        $div.append(catContainer);
+        Messages.admin_support_premium = "Premium tickets:"; // XXX
+        Messages.admin_support_normal = "Unanswered tickets:";
+        Messages.admin_support_answered = "Answered tickets:";
+        Messages.admin_support_closed = "Closed tickets:";
+        Messages.admin_support_open = "Show";
+        Messages.admin_support_collapse = "Collapse";
+        Messages.admin_support_first = "Created on: ";
+        Messages.admin_support_last = "Updated on: ";
+        var col1 = h('div.cp-support-column', h('h1', [
+            h('span', Messages.admin_support_premium),
+            h('span.cp-support-count'),
+            h('button.btn.cp-support-column-button', Messages.admin_support_collapse)
+        ]));
+        var col2 = h('div.cp-support-column', h('h1', [
+            h('span', Messages.admin_support_normal),
+            h('span.cp-support-count'),
+            h('button.btn.cp-support-column-button', Messages.admin_support_collapse)
+        ]));
+        var col3 = h('div.cp-support-column', h('h1', [
+            h('span', Messages.admin_support_answered),
+            h('span.cp-support-count'),
+            h('button.btn.cp-support-column-button', Messages.admin_support_collapse)
+        ]));
+        var col4 = h('div.cp-support-column', h('h1', [
+            h('span', Messages.admin_support_closed),
+            h('span.cp-support-count'),
+            h('button.btn.cp-support-column-button', Messages.admin_support_collapse)
+        ]));
+        var $col1 = $(col1), $col2 = $(col2), $col3 = $(col3), $col4 = $(col4);
+        $div.append([
+            //catContainer
+            col1,
+            col2,
+            col3,
+            col4
+        ]);
+        $div.find('.cp-support-column-button').click(function () {
+            var $col = $(this).closest('.cp-support-column');
+            $col.toggleClass('cp-support-column-collapsed');
+            if ($col.hasClass('cp-support-column-collapsed')) {
+                $(this).text(Messages.admin_support_open);
+                $(this).toggleClass('btn-primary');
+            } else {
+                $(this).text(Messages.admin_support_collapse);
+                $(this).toggleClass('btn-primary');
+            }
+        });
         var category = 'all';
         var $drop = APP.support.makeCategoryDropdown(catContainer, function (key) {
             category = key;
@@ -447,37 +618,128 @@ define([
 
         var hashesById = {};
 
-        var reorder = function () {
+        var getTicketData = function (id) {
-            var order = Object.keys(hashesById);
+            var t = hashesById[id];
-            order.sort(function (id1, id2) {
+            if (!Array.isArray(t) || !t.length) { return; }
-                var t1 = hashesById[id1];
+            var ed = Util.find(t[0], ['content', 'msg', 'content', 'sender', 'edPublic']);
-                var t2 = hashesById[id2];
+            // If one of their ticket was sent as a premium user, mark them as premium
-                if (!Array.isArray(t1)) { return 1; }
+            var premium = t.some(function (msg) {
-                if (!Array.isArray(t2)) { return -1; }
+                var _ed = Util.find(msg, ['content', 'msg', 'content', 'sender', 'edPublic']);
-                var lastMsg1 = t1[t1.length - 1];
+                if (ed !== _ed) { return; }
-                var lastMsg2 = t2[t2.length - 1];
+                return Util.find(t[0], ['content', 'msg', 'content', 'sender', 'plan']);
-                var time1 = Util.find(lastMsg1, ['content', 'msg', 'content', 'time']);
+            });
-                var time2 = Util.find(lastMsg2, ['content', 'msg', 'content', 'time']);
+            var lastMsg = t[t.length - 1];
-                var authorEd1 = Util.find(lastMsg1, ['content', 'msg', 'content', 'sender', 'edPublic']);
+            var lastMsgEd = Util.find(lastMsg, ['content', 'msg', 'content', 'sender', 'edPublic']);
-                var authorEd2 = Util.find(lastMsg2, ['content', 'msg', 'content', 'sender', 'edPublic']);
+            return {
-                var admin1 = ApiConfig.adminKeys.indexOf(authorEd1) !== -1;
+                lastMsg: lastMsg,
-                var admin2 = ApiConfig.adminKeys.indexOf(authorEd2) !== -1;
+                time: Util.find(lastMsg, ['content', 'msg', 'content', 'time']),
-                // If one is answered and not the other, put the unanswered first
+                lastMsgEd: lastMsgEd,
-                if (admin1 && !admin2) { return 1; }
+                lastAdmin: lastMsgEd !== ed && ApiConfig.adminKeys.indexOf(lastMsgEd) !== -1,
-                if (!admin1 && admin2) { return -1; }
+                premium: premium,
-                // Otherwise, sort them by time
+                authorEd: ed,
-                return time2 - time1;
+                closed: Util.find(lastMsg, ['content', 'msg', 'type']) === 'CLOSE'
+            };
+        };
+
+        var addClickHandler = function ($ticket) {
+            $ticket.on('click', function () {
+                $ticket.toggleClass('cp-support-open', true);
+                $ticket.off('click');
+            });
+        };
+        var makeOpenButton = function ($ticket) {
+            var button = h('button.btn.btn-primary.cp-support-expand', Messages.admin_support_open);
+            var collapse = h('button.btn.cp-support-collapse', Messages.admin_support_collapse);
+            $(button).click(function () {
+                $ticket.toggleClass('cp-support-open', true);
+            });
+            addClickHandler($ticket);
+            $(collapse).click(function (e) {
+                $ticket.toggleClass('cp-support-open', false);
+                e.stopPropagation();
+                setTimeout(function () {
+                    addClickHandler($ticket);
+                });
             });
-            order.forEach(function (id, i) {
+            $ticket.find('.cp-support-title-buttons').prepend([button, collapse]);
-                $div.find('[data-id="'+id+'"]').css('order', i);
+            $ticket.append(h('div.cp-support-collapsed'));
+        };
+        var updateTicketDetails = function ($ticket, isPremium) {
+            var $first = $ticket.find('.cp-support-message-from').first();
+            var user = $first.find('span').first().html();
+            var time = $first.find('.cp-support-message-time').text();
+            var last = $ticket.find('.cp-support-message-from').last().find('.cp-support-message-time').text();
+            var $c = $ticket.find('.cp-support-collapsed');
+            var txtClass = isPremium ? ".cp-support-ispremium" : "";
+            $c.html('').append([
+                UI.setHTML(h('span'+ txtClass), user),
+                h('span', [
+                    h('b', Messages.admin_support_first),
+                    h('span', time)
+                ]),
+                h('span', [
+                    h('b', Messages.admin_support_last),
+                    h('span', last)
+                ])
+            ]);
+
+        };
+
+        var sort = function (id1, id2) {
+            var t1 = getTicketData(id1);
+            var t2 = getTicketData(id2);
+            if (!t1) { return 1; }
+            if (!t2) { return -1; }
+            /*
+            // If one is answered and not the other, put the unanswered first
+            if (t1.lastAdmin && !t2.lastAdmin) { return 1; }
+            if (!t1.lastAdmin && t2.lastAdmin) { return -1; }
+            */
+            // Otherwise, sort them by time
+            return t1.time - t2.time;
+        };
+
+        var _reorder = function () {
+            var orderAnswered = Object.keys(hashesById).filter(function (id) {
+                var d = getTicketData(id);
+                return d && d.lastAdmin && !d.closed;
+            }).sort(sort);
+            var orderPremium = Object.keys(hashesById).filter(function (id) {
+                var d = getTicketData(id);
+                return d && d.premium && !d.lastAdmin && !d.closed;
+            }).sort(sort);
+            var orderNormal = Object.keys(hashesById).filter(function (id) {
+                var d = getTicketData(id);
+                return d && !d.premium && !d.lastAdmin && !d.closed;
+            }).sort(sort);
+            var orderClosed = Object.keys(hashesById).filter(function (id) {
+                var d = getTicketData(id);
+                return d && d.closed;
+            }).sort(sort);
+            var cols = [$col1, $col2, $col3, $col4];
+            [orderPremium, orderNormal, orderAnswered, orderClosed].forEach(function (list, j) {
+                list.forEach(function (id, i) {
+                    var $t = $div.find('[data-id="'+id+'"]');
+                    var d = getTicketData(id);
+                    $t.css('order', i).appendTo(cols[j]);
+                    updateTicketDetails($t, d.premium);
+                });
+                if (!list.length) {
+                    cols[j].hide();
+                } else {
+                    cols[j].show();
+                    cols[j].find('.cp-support-count').text(list.length);
+                }
             });
         };
+        var reorder = Util.throttle(_reorder, 150);
 
         var to = Util.throttle(function () {
             var $ticket = $div.find('.cp-support-list-ticket[data-id="'+linkedId+'"]');
+            $ticket.addClass('cp-support-open');
             $ticket[0].scrollIntoView();
             linkedId = undefined;
-        }, 100);
+        }, 200);
 
         // Register to the "support" mailbox
         common.mailbox.subscribe(['supportadmin'], {
@@ -505,6 +767,7 @@ define([
                     if (!$ticket.length) { return; }
                     $ticket.addClass('cp-support-list-closed');
                     $ticket.append(APP.support.makeCloseMessage(content, hash));
+                    reorder();
                     return;
                 }
                 if (msg.type !== 'TICKET') { return; }
@@ -525,13 +788,19 @@ define([
                                 }));
                             });
                         }).nThen(function () {
-                            if (!error) { return void $ticket.remove(); }
+                            if (!error) {
+                                $ticket.remove();
+                                delete hashesById[id];
+                                reorder();
+                                return;
+                            }
                             // if deletion failed then reactivate the button and warn
                             hideButton.removeAttribute('disabled');
                             // and show a generic error message
                             UI.alert(Messages.error);
                         });
                     });
+                    makeOpenButton($ticket);
                     if (category !== 'all' && $ticket.attr('data-cat') !== category) {
                         $ticket.hide();
                     }

www/assert/main.js

@@ -334,6 +334,12 @@ define([
             !secret.hashData.present);
     }, "test support for ugly tracking query paramaters in url");
 
+    assert(function (cb) {
+        var url = '//cryptpad.fr/pad/#/2/pad/edit/oRE0oLCtEXusRDyin7GyLGcS/';
+        var parsed = Hash.isValidHref(url);
+        cb(!parsed);
+    }, "test that protocol relative URLs are rejected");
+
     assert(function (cb) {
         var keys = Block.genkeys(Nacl.randomBytes(64));
         var hash = Block.getBlockHash(keys);
@@ -349,7 +355,7 @@ define([
         var v3 = Hash.isValidHref('/pad');
         var v4 = Hash.isValidHref('/pad/');
 
-        var res = v1 && v2 && v3 && v4;
+        var res = Boolean(v1 && v2 && v3 && v4);
         cb(res);
         if (!res) {
             console.log(v1, v2, v3, v4);
@@ -361,7 +367,7 @@ define([
         var v3 = Hash.isValidHref('/pad#'); // Invalid
         var v4 = Hash.isValidHref('/pad/#');
 
-        var res = v1 && v2 && v3 && v4;
+        var res = Boolean(v1 && v2 && v3 && v4);
         cb(res);
         if (!res) {
             console.log(v1, v2, v3, v4);
@@ -373,7 +379,7 @@ define([
         var v3 = Hash.isValidHref('https://cryptpad.fr/pad/#67b8385b07352be53e40746d2be6ccd7XAYSuJYYqa9NfmInyHci7LNy');
         var v4 = Hash.isValidHref('/pad/#/2/pad/edit/HGu0tK2od-2BBnwAz2ZNS-t4/p/embed');
 
-        var res = v1 && v2 && v3 && v4;
+        var res = Boolean(v1 && v2 && v3 && v4);
         cb(res);
         if (!res) {
             console.log(v1, v2, v3, v4);

www/common/application_config_internal.js

@@ -169,7 +169,7 @@ define(function() {
     // make them have a very slow loading time. To avoid impacting the user experience
     // significantly, we're limiting the number of teams per user to 3 by default.
     // You can change this value here.
-    //config.maxTeamsSlots = 3;
+    //config.maxTeamsSlots = 5;
 
     // Each team is considered as a registered user by the server. Users and teams are indistinguishable
     // in the database so teams will offer the same storage limits as users by default.
@@ -177,7 +177,7 @@ define(function() {
     // We're limiting the number of teams each user is able to own to 1 in order to make sure
     // users don't use "fake" teams (1 member) just to increase their storage limit.
     // You can change the value here.
-    // config.maxOwnedTeams = 1;
+    // config.maxOwnedTeams = 5;
 
     return config;
 });

www/common/common-constants.js

@@ -12,8 +12,8 @@ define(['/customize/application_config.js'], function (AppConfig) {
         tokenKey: 'loginToken',
         displayPadCreationScreen: 'displayPadCreationScreen',
         deprecatedKey: 'deprecated',
-        MAX_TEAMS_SLOTS: AppConfig.maxTeamsSlots || 3,
+        MAX_TEAMS_SLOTS: AppConfig.maxTeamsSlots || 5,
-        MAX_TEAMS_OWNED: AppConfig.maxOwnedTeams || 1,
+        MAX_TEAMS_OWNED: AppConfig.maxOwnedTeams || 5,
         // Apps
         criticalApps: ['profile', 'settings', 'debug', 'admin', 'support', 'notifications']
     };

www/common/common-hash.js

@@ -464,6 +464,8 @@ Version 4: Data URL when not a realtime link yet (new pad or "static" app)
         };
 
         if (!/^https*:\/\//.test(href)) {
+            // If it doesn't start with http(s), it should be a relative href
+            if (!/^\/($|[^\/])/.test(href)) { return ret; }
             idx = href.indexOf('/#');
             ret.type = href.slice(1, idx);
             if (idx === -1) { return ret; }
@@ -661,7 +663,7 @@ Version 4: Data URL when not a realtime link yet (new pad or "static" app)
                 if (parsed.hashData.key && !/^[a-zA-Z0-9+-/=]+$/.test(parsed.hashData.key)) { return; }
             }
         }
-        return true;
+        return parsed;
     };
 
     Hash.decodeDataOptions = function (opts) {

www/common/common-interface.js

@@ -65,12 +65,13 @@ define([
             switch (e.which) {
                 case 27: // cancel
                     if (typeof(no) === 'function') { no(e); }
+                    $(el || window).off('keydown', handler);
                     break;
                 case 13: // enter
                     if (typeof(yes) === 'function') { yes(e); }
+                    $(el || window).off('keydown', handler);
                     break;
             }
-            $(el || window).off('keydown', handler);
         };
 
         $(el || window).keydown(handler);
@@ -773,7 +774,8 @@ define([
             $(originalBtn).show();
         };
 
-        $button.click(function () {
+        $button.click(function (e) {
+            e.stopPropagation();
             done(true);
         });
 
@@ -791,7 +793,8 @@ define([
             to = setTimeout(todo, INTERVAL);
         };
 
-        $(originalBtn).addClass('cp-button-confirm-placeholder').click(function () {
+        $(originalBtn).addClass('cp-button-confirm-placeholder').click(function (e) {
+            e.stopPropagation();
             // If we have a validation function, continue only if it's true
             if (config.validate && !config.validate()) { return; }
             i = 1;
@@ -981,6 +984,11 @@ define([
         setTimeout(cb, 750);
     };
     UI.errorLoadingScreen = function (error, transparent, exitable) {
+        if (error === 'Error: XDR encoding failure') {
+            console.warn(error);
+            return;
+        }
+
         var $loading = $('#' + LOADING);
         if (!$loading.is(':visible') || $loading.hasClass('cp-loading-hidden')) {
             UI.addLoadingScreen();

www/common/common-messaging.js

@@ -91,7 +91,7 @@ define([
     Msg.updateMyData = function (store, curve) {
         var myData = createData(store.proxy, false);
         if (store.proxy.friends) {
-            store.proxy.friends.me = myData;
+            store.proxy.friends.me = Util.clone(myData);
             delete store.proxy.friends.me.channel;
         }
         if (store.modules['team']) {
@@ -99,6 +99,7 @@ define([
         }
         var todo = function (friend) {
             if (!friend || !friend.notifications) { return; }
+            delete friend.user;
             myData.channel = friend.channel;
             store.mailbox.sendTo('UPDATE_DATA', myData, {
                 channel: friend.notifications,

www/common/common-thumbnail.js

@@ -3,9 +3,9 @@ define([
     '/common/common-util.js',
     '/common/visible.js',
     '/common/common-hash.js',
-    '/file/file-crypto.js',
+    '/common/media-tag.js',
     '/bower_components/tweetnacl/nacl-fast.min.js',
-], function ($, Util, Visible, Hash, FileCrypto) {
+], function ($, Util, Visible, Hash, MediaTag) {
     var Nacl = window.nacl;
     var Thumb = {
         dimension: 100,
@@ -314,7 +314,7 @@ define([
             var hexFileName = secret.channel;
             var src = fileHost + Hash.getBlobPathFromHex(hexFileName);
             var key = secret.keys && secret.keys.cryptKey;
-            FileCrypto.fetchDecryptedMetadata(src, key, function (e, metadata) {
+            MediaTag.fetchDecryptedMetadata(src, key, function (e, metadata) {
                 if (e) {
                     if (e === 'XHR_ERROR') { return; }
                     return console.error(e);

www/common/common-ui-elements.js

@@ -1602,9 +1602,9 @@ define([
                 content: h('span', Messages.profileButton),
                 action: function () {
                     if (padType) {
-                        window.open(origin+'/profile/');
+                        Common.openURL(origin+'/profile/');
                     } else {
-                        window.parent.location = origin+'/profile/';
+                        Common.gotoURL(origin+'/profile/');
                     }
                 },
             });
@@ -1613,33 +1613,36 @@ define([
             options.push({
                 tag: 'a',
                 attributes: {
-                    'target': '_blank',
-                    'href': origin+'/drive/',
                     'class': 'fa fa-hdd-o'
                 },
-                content: h('span', Messages.type.drive)
+                content: h('span', Messages.type.drive),
+                action: function () {
+                    Common.openURL(origin+'/drive/');
+                },
             });
         }
         if (padType !== 'teams' && accountName) {
             options.push({
                 tag: 'a',
                 attributes: {
-                    'target': '_blank',
-                    'href': origin+'/teams/',
                     'class': 'fa fa-users'
                 },
-                content: h('span', Messages.type.teams)
+                content: h('span', Messages.type.teams),
+                action: function () {
+                    Common.openURL('/teams/');
+                },
             });
         }
         if (padType !== 'contacts' && accountName) {
             options.push({
                 tag: 'a',
                 attributes: {
-                    'target': '_blank',
-                    'href': origin+'/contacts/',
                     'class': 'fa fa-address-book'
                 },
-                content: h('span', Messages.type.contacts)
+                content: h('span', Messages.type.contacts),
+                action: function () {
+                    Common.openURL('/contacts/');
+                },
             });
         }
         if (padType !== 'settings') {
@@ -1649,9 +1652,9 @@ define([
                 content: h('span', Messages.settingsButton),
                 action: function () {
                     if (padType) {
-                        window.open(origin+'/settings/');
+                        Common.openURL(origin+'/settings/');
                     } else {
-                        window.parent.location = origin+'/settings/';
+                        Common.gotoURL(origin+'/settings/');
                     }
                 },
             });
@@ -1666,9 +1669,9 @@ define([
                 content: h('span', Messages.adminPage || 'Admin'),
                 action: function () {
                     if (padType) {
-                        window.open(origin+'/admin/');
+                        Common.openURL(origin+'/admin/');
                     } else {
-                        window.parent.location = origin+'/admin/';
+                        Common.gotoURL(origin+'/admin/');
                     }
                 },
             });
@@ -1690,29 +1693,28 @@ define([
                 content: h('span', Messages.supportPage || 'Support'),
                 action: function () {
                     if (padType) {
-                        window.open(origin+'/support/');
+                        Common.openURL(origin+'/support/');
                     } else {
-                        window.parent.location = origin+'/support/';
+                        Common.gotoURL(origin+'/support/');
                     }
                 },
             });
         }
-        // XXX Trade the survey for documentation
+/*
-        // if (AppConfig.surveyURL) {
+        if (AppConfig.surveyURL) {
-        //     options.push({
+            options.push({
-        //         tag: 'a',
+                tag: 'a',
-        //         attributes: {
+                attributes: {
-        //             'target': '_blank',
+                    'class': 'cp-toolbar-survey fa fa-graduation-cap'
-        //             'rel': 'noopener',
+                },
-        //             'href': AppConfig.surveyURL,
+                content: h('span', Messages.survey),
-        //             'class': 'cp-toolbar-survey fa fa-graduation-cap'
+                action: function () {
-        //         },
+                    Common.openUnsafeURL(AppConfig.surveyURL);
-        //         content: h('span', Messages.survey),
+                    Feedback.send('SURVEY_CLICKED');
-        //         action: function () {
+                },
-        //             Feedback.send('SURVEY_CLICKED');
+            });
-        //         },
+        }
-        //     });
+*/
-        // }
         options.push({
             tag: 'a',
             attributes: {
@@ -1727,11 +1729,12 @@ define([
         options.push({
             tag: 'a',
             attributes: {
-                'target': '_blank',
-                'href': origin+'/index.html',
                 'class': 'fa fa-home'
             },
-            content: h('span', Messages.homePage)
+            content: h('span', Messages.homePage),
+            action: function () {
+                Common.openURL('/index.html');
+            },
         });
         // Add the change display name button if not in read only mode
         /*
@@ -1747,23 +1750,24 @@ define([
             options.push({
                 tag: 'a',
                 attributes: {
-                    'target': '_blank',
-                    'href': priv.plan ? priv.accounts.upgradeURL : origin+'/features.html',
                     'class': 'fa fa-star-o'
                 },
-                content: h('span', priv.plan ? Messages.settings_cat_subscription : Messages.pricing)
+                content: h('span', priv.plan ? Messages.settings_cat_subscription : Messages.pricing),
+                action: function () {
+                    Common.openURL(priv.plan ? priv.accounts.upgradeURL :'/features.html');
+                },
             });
         }
         if (!priv.plan && !Config.removeDonateButton) {
             options.push({
                 tag: 'a',
                 attributes: {
-                    'target': '_blank',
-                    'rel': 'noopener',
-                    'href': priv.accounts.donateURL,
                     'class': 'fa fa-gift'
                 },
-                content: h('span', Messages.crowdfunding_button2)
+                content: h('span', Messages.crowdfunding_button2),
+                action: function () {
+                    Common.openUnsafeURL(priv.accounts.donateURL);
+                },
             });
         }
 
@@ -1778,7 +1782,7 @@ define([
                 content: h('span', Messages.logoutEverywhere),
                 action: function () {
                     Common.getSframeChannel().query('Q_LOGOUT_EVERYWHERE', null, function () {
-                        window.parent.location = origin + '/';
+                        Common.gotoURL(origin + '/');
                     });
                 },
             });
@@ -1788,7 +1792,7 @@ define([
                 content: h('span', Messages.logoutButton),
                 action: function () {
                     Common.logout(function () {
-                        window.parent.location = origin+'/';
+                        Common.gotoURL(origin+'/');
                     });
                 },
             });
@@ -2603,7 +2607,9 @@ define([
 
         var block = h('div#cp-loading-burn-after-reading', [
             info,
-            button
+            h('nav', {
+                style: 'text-align: right'
+            }, button),
         ]);
         UI.errorLoadingScreen(block);
     };
@@ -2738,6 +2744,43 @@ define([
 
     };
 
+    UIElements.displayTrimHistoryPrompt = function (common, data) {
+        var mb = Util.bytesToMegabytes(data.size);
+        var text = Messages._getKey('history_trimPrompt', [
+            Messages._getKey('formattedMB', [mb])
+        ]);
+        var yes = h('button.cp-corner-primary', [
+            h('span.fa.fa-trash-o'),
+            Messages.trimHistory_button
+        ]);
+        var no = h('button.cp-corner-cancel', Messages.crowdfunding_popup_no); // Not now
+        var actions = h('div', [no, yes]);
+
+        var dontShowAgain = function () {
+            var until = (+new Date()) + (7 * 24 * 3600 * 1000); // 7 days from now
+            if (data.drive) {
+                common.setAttribute(['drive', 'trim'], until);
+                return;
+            }
+            common.setPadAttribute('trim', until);
+        };
+
+        var modal = UI.cornerPopup(text, actions, '', {});
+
+        $(yes).click(function () {
+            modal.delete();
+            if (data.drive) {
+                common.openURL('/settings/#drive');
+                return;
+            }
+            common.getSframeChannel().event('EV_PROPERTIES_OPEN');
+        });
+        $(no).click(function () {
+            dontShowAgain();
+            modal.delete();
+        });
+    };
+
     UIElements.displayFriendRequestModal = function (common, data) {
         var msg = data.content.msg;
         var userData = msg.content.user;

www/common/common-util.js

@@ -30,6 +30,15 @@
         return JSON.parse(JSON.stringify(o));
     };
 
+    Util.serializeError = function (err) {
+        if (!(err instanceof Error)) { return err; }
+        var ser = {};
+        Object.getOwnPropertyNames(err).forEach(function (key) {
+            ser[key] = err[key];
+        });
+        return ser;
+    };
+
     Util.tryParse = function (s) {
         try { return JSON.parse(s); } catch (e) { return;}
     };
@@ -113,13 +122,13 @@
         var handle = function (id, args) {
             var fn = pending[id];
             if (typeof(fn) !== 'function') {
-                errorHandler("MISSING_CALLBACK", {
+                return void errorHandler("MISSING_CALLBACK", {
                     id: id,
                     args: args,
                 });
             }
             try {
-                pending[id].apply(null, Array.isArray(args)? args : [args]);
+                fn.apply(null, Array.isArray(args)? args : [args]);
             } catch (err) {
                 errorHandler('HANDLER_ERROR', {
                     error: err,
@@ -265,28 +274,73 @@
 
 
     // given a path, asynchronously return an arraybuffer
-    Util.fetch = function (src, cb, progress) {
+    var getCacheKey = function (src) {
-        var CB = Util.once(cb);
+        var _src = src.replace(/(\/)*$/, ''); // Remove trailing slashes
+        var idx = _src.lastIndexOf('/');
+        var cacheKey = _src.slice(idx+1);
+        if (!/^[a-f0-9]{48}$/.test(cacheKey)) { cacheKey = undefined; }
+        return cacheKey;
+    };
+    Util.fetch = function (src, cb, progress, cache) {
+        var CB = Util.once(Util.mkAsync(cb));
+
+        var cacheKey = getCacheKey(src);
+        var getBlobCache = function (id, cb) {
+            if (!cache || typeof(cache.getBlobCache) !== "function") { return void cb('EINVAL'); }
+            cache.getBlobCache(id, cb);
+        };
+        var setBlobCache = function (id, u8, cb) {
+            if (!cache || typeof(cache.setBlobCache) !== "function") { return void cb('EINVAL'); }
+            cache.setBlobCache(id, u8, cb);
+        };
 
-        var xhr = new XMLHttpRequest();
+        var xhr;
-        xhr.open("GET", src, true);
+
-        if (progress) {
+        var fetch = function () {
-            xhr.addEventListener("progress", function (evt) {
+            xhr = new XMLHttpRequest();
-                if (evt.lengthComputable) {
+            xhr.open("GET", src, true);
-                    var percentComplete = evt.loaded / evt.total;
+            if (progress) {
-                    progress(percentComplete);
+                xhr.addEventListener("progress", function (evt) {
+                    if (evt.lengthComputable) {
+                        var percentComplete = evt.loaded / evt.total;
+                        progress(percentComplete);
+                    }
+                }, false);
+            }
+            xhr.responseType = "arraybuffer";
+            xhr.onerror = function (err) { CB(err); };
+            xhr.onload = function () {
+                if (/^4/.test(''+this.status)) {
+                    return CB('XHR_ERROR');
                 }
-            }, false);
+
-        }
+                var arrayBuffer = xhr.response;
-        xhr.responseType = "arraybuffer";
+                if (arrayBuffer) {
-        xhr.onerror = function (err) { CB(err); };
+                    var u8 = new Uint8Array(arrayBuffer);
-        xhr.onload = function () {
+                    if (cacheKey) {
-            if (/^4/.test(''+this.status)) {
+                        return void setBlobCache(cacheKey, u8, function () {
-                return CB('XHR_ERROR');
+                            CB(null, u8);
+                        });
+                    }
+                    return void CB(void 0, u8);
+                }
+                CB('ENOENT');
+            };
+            xhr.send(null);
+        };
+
+        if (!cacheKey) { return void fetch(); }
+
+        getBlobCache(cacheKey, function (err, u8) {
+            if (err || !u8) { return void fetch(); }
+            CB(void 0, u8);
+        });
+
+        return {
+            cancel: function () {
+                if (xhr && xhr.abort) { xhr.abort(); }
             }
-            return void CB(void 0, new Uint8Array(xhr.response));
         };
-        xhr.send(null);
     };
 
     Util.dataURIToBlob = function (dataURI) {

www/common/cryptget.js

@@ -6,10 +6,11 @@ define([
     '/common/common-hash.js',
     '/common/common-realtime.js',
     '/common/outer/network-config.js',
+    '/common/outer/cache-store.js',
     '/common/pinpad.js',
     '/bower_components/nthen/index.js',
     '/bower_components/chainpad/chainpad.dist.js',
-], function (Crypto, CPNetflux, Netflux, Util, Hash, Realtime, NetConfig, Pinpad, nThen) {
+], function (Crypto, CPNetflux, Netflux, Util, Hash, Realtime, NetConfig, Cache, Pinpad, nThen) {
     var finish = function (S, err, doc) {
         if (S.done) { return; }
         S.cb(err, doc);
@@ -92,7 +93,8 @@ define([
             validateKey: secret.keys.validateKey || undefined,
             crypto: Crypto.createEncryptor(secret.keys),
             logLevel: 0,
-            initialState: opt.initialState
+            initialState: opt.initialState,
+            Cache: Cache
         };
         return config;
     };
@@ -132,9 +134,11 @@ define([
         };
 
         config.onError = function (info) {
+            console.warn(info);
             finish(Session, info.error);
         };
         config.onChannelError = function (info) {
+            console.error(info);
             finish(Session, info.error);
         };
 

www/common/cryptpad-common.js

@@ -3,6 +3,7 @@ define([
     '/customize/messages.js',
     '/common/common-util.js',
     '/common/common-hash.js',
+    '/common/outer/cache-store.js',
     '/common/common-messaging.js',
     '/common/common-constants.js',
     '/common/common-feedback.js',
@@ -14,7 +15,7 @@ define([
 
     '/customize/application_config.js',
     '/bower_components/nthen/index.js',
-], function (Config, Messages, Util, Hash,
+], function (Config, Messages, Util, Hash, Cache,
             Messaging, Constants, Feedback, Visible, UserObject, LocalStore, Channel, Block,
             AppConfig, Nthen) {
 
@@ -147,6 +148,22 @@ define([
         };
         send();
     };
+    common.fixRosterHash = function () {
+        // Push teams keys
+        postMessage("GET", {
+            key: ['teams'],
+        }, function (obj) {
+            if (obj.error) { return console.error(obj.error); }
+            Object.keys(obj || {}).forEach(function (id) {
+                postMessage("SET", {
+                    key: ['teams', id, 'keys', 'roster', 'lastKnownHash'],
+                    value: ''
+                }, function () {
+                    console.log('done, please close all your CryptPad tabs before testing the fix');
+                });
+            });
+        });
+    };
 
     (function () {
         var bypassHashChange = function (key) {
@@ -701,7 +718,7 @@ define([
         });
     };
 
-    common.useFile = function (Crypt, cb, optsPut) {
+    common.useFile = function (Crypt, cb, optsPut, onProgress) {
         var fileHost = Config.fileHost || window.location.origin;
         var data = common.fromFileData;
         var parsed = Hash.parsePadUrl(data.href);
@@ -758,7 +775,9 @@ define([
                         return void cb(err);
                     }
                     u8 = _u8;
-                }));
+                }), function (progress) {
+                    onProgress(progress * 50);
+                }, Cache);
             }).nThen(function (waitFor) {
                 require(["/file/file-crypto.js"], waitFor(function (FileCrypto) {
                     FileCrypto.decrypt(u8, key, waitFor(function (err, _res) {
@@ -767,7 +786,9 @@ define([
                             return void cb(err);
                         }
                         res = _res;
-                    }));
+                    }), function (progress) {
+                        onProgress(50 + progress * 50);
+                    });
                 }));
             }).nThen(function (waitFor) {
                 var ext = Util.parseFilename(data.title).ext;
@@ -991,6 +1012,8 @@ define([
     pad.onJoinEvent = Util.mkEvent();
     pad.onLeaveEvent = Util.mkEvent();
     pad.onDisconnectEvent = Util.mkEvent();
+    pad.onCacheEvent = Util.mkEvent();
+    pad.onCacheReadyEvent = Util.mkEvent();
     pad.onConnectEvent = Util.mkEvent();
     pad.onErrorEvent = Util.mkEvent();
     pad.onMetadataEvent = Util.mkEvent();
@@ -1003,6 +1026,10 @@ define([
         postMessage("GIVE_PAD_ACCESS", data, cb);
     };
 
+    common.onCorruptedCache = function (channel) {
+        postMessage("CORRUPTED_CACHE", channel);
+    };
+
     common.setPadMetadata = function (data, cb) {
         postMessage('SET_PAD_METADATA', data, cb);
     };
@@ -1876,12 +1903,12 @@ define([
 
     var requestLogin = function () {
         // log out so that you don't go into an endless loop...
-        LocalStore.logout();
+        LocalStore.logout(function () {
-
+            // redirect them to log in, and come back when they're done.
-        // redirect them to log in, and come back when they're done.
+            var href = Hash.hashToHref('', 'login');
-        var href = Hash.hashToHref('', 'login');
+            var url = Hash.getNewPadURL(href, { href: currentPad.href });
-        var url = Hash.getNewPadURL(href, { href: currentPad.href });
+            window.location.href = url;
-        window.location.href = url;
+        });
     };
 
     common.startAccountDeletion = function (data, cb) {
@@ -1956,6 +1983,8 @@ define([
         PAD_JOIN: common.padRpc.onJoinEvent.fire,
         PAD_LEAVE: common.padRpc.onLeaveEvent.fire,
         PAD_DISCONNECT: common.padRpc.onDisconnectEvent.fire,
+        PAD_CACHE: common.padRpc.onCacheEvent.fire,
+        PAD_CACHE_READY: common.padRpc.onCacheReadyEvent.fire,
         PAD_CONNECT: common.padRpc.onConnectEvent.fire,
         PAD_ERROR: common.padRpc.onErrorEvent.fire,
         PAD_METADATA: common.padRpc.onMetadataEvent.fire,
@@ -2057,6 +2086,32 @@ define([
 
         var userHash;
 
+        (function iOSFirefoxFix () {
+/*
+    For some bizarre reason Firefox on iOS throws an error during the
+    loading process unless we call this function. Drawing these elements
+    to the DOM presumably causes the JS engine to wait just a little bit longer
+    until some APIs we need are ready. This occurs despite all this code being
+    run after the usual dom-ready events. This fix was discovered while trying
+    to log the error messages to the DOM because it's extremely difficult
+    to debug Firefox iOS in the usual ways. In summary, computers are terrible.
+*/
+             try {
+                var style = document.createElement('style');
+                    style.type = 'text/css';
+                    style.appendChild(document.createTextNode('#cp-logger { display: none; }'));
+                document.head.appendChild(style);
+
+                var logger = document.createElement('div');
+                    logger.setAttribute('id', 'cp-logger');
+                document.body.appendChild(logger);
+
+                var pre = document.createElement('pre');
+                    pre.innerText = 'x';
+                logger.appendChild(pre);
+            } catch (err) { console.error(err); }
+        }());
+
         Nthen(function (waitFor) {
             if (AppConfig.beforeLogin) {
                 AppConfig.beforeLogin(LocalStore.isLoggedIn(), waitFor());

www/common/diffMarked.js

@@ -303,14 +303,22 @@ define([
         return renderParagraph(p);
     };
 
+    // Note: iframe, video and audio are used in mediatags and are allowed in rich text pads.
     var forbiddenTags = [
         'SCRIPT',
-        'IFRAME',
+        //'IFRAME',
         'OBJECT',
         'APPLET',
-        'VIDEO', // privacy implications of videos are the same as images
+        //'VIDEO', // privacy implications of videos are the same as images
-        'AUDIO', // same with audio
+        //'AUDIO', // same with audio
+        'SOURCE'
+    ];
+    var restrictedTags = [
+        'IFRAME',
+        'VIDEO',
+        'AUDIO'
     ];
+
     var unsafeTag = function (info) {
         /*if (info.node && $(info.node).parents('media-tag').length) {
             // Do not remove elements inside a media-tag
@@ -343,13 +351,20 @@ define([
         if (!(node && node.parentElement)) { return; }
         var parent = node.parentElement;
         if (!parent) { return; }
-        console.log('removing %s tag', node.nodeName);
+        console.debug('removing %s tag', node.nodeName);
         parent.removeChild(node);
     };
 
+    // Only allow iframe, video and audio with local source
+    var checkSrc = function (root) {
+        if (restrictedTags.indexOf(root.nodeName.toUpperCase()) === -1) { return true; }
+        return root.getAttribute && /^(blob\:|\/common\/pdfjs)/.test(root.getAttribute('src'));
+    };
+
     var removeForbiddenTags = function (root) {
         if (!root) { return; }
         if (forbiddenTags.indexOf(root.nodeName.toUpperCase()) !== -1) { removeNode(root); }
+        if (!checkSrc(root)) { removeNode(root); }
         slice(root.children).forEach(removeForbiddenTags);
     };
 
@@ -658,7 +673,7 @@ define([
                     $(contextMenu.menu).find('li').show();
                     contextMenu.show(e);
                 });
-                if ($mt.children().length) {
+                if ($mt.children().length && $mt[0]._mediaObject) {
                     $mt.off('click dblclick preview');
                     $mt.on('preview', onPreview($mt));
                     if ($mt.find('img').length) {
@@ -668,15 +683,15 @@ define([
                     }
                     return;
                 }
-                MediaTag(el);
+                var mediaObject = MediaTag(el);
                 var observer = new MutationObserver(function(mutations) {
                     mutations.forEach(function(mutation) {
                         if (mutation.type === 'childList') {
-                            var list_values = slice(mutation.target.children)
+                            var list_values = slice(el.children)
                                                 .map(function (el) { return el.outerHTML; })
                                                 .join('');
-                            mediaMap[mutation.target.getAttribute('src')] = list_values;
+                            mediaMap[el.getAttribute('src')] = list_values;
-                            observer.disconnect();
+                            if (mediaObject.complete) { observer.disconnect(); }
                         }
                     });
                     $mt.off('click dblclick preview');
@@ -689,6 +704,7 @@ define([
                 });
                 observer.observe(el, {
                     attributes: false,
+                    subtree: true,
                     childList: true,
                     characterData: false
                 });

www/common/drive-ui.js

@@ -42,6 +42,7 @@ define([
 
     var APP = window.APP = {
         editable: false,
+        online: true,
         mobile: function () {
             if (window.matchMedia) { return !window.matchMedia('(any-pointer:fine)').matches; }
             else { return $('body').width() <= 600; }
@@ -267,13 +268,25 @@ define([
     };
 
     // Handle disconnect/reconnect
-    var setEditable = function (state, isHistory) {
+    // If isHistory and isSf are both false, update the "APP.online" flag
+    // If isHistory is true, update the "APP.history" flag
+    // isSf is used to detect offline shared folders: setEditable is called on displayDirectory
+    var setEditable = function (state, isHistory, isSf) {
         if (APP.closed || !APP.$content || !$.contains(document.documentElement, APP.$content[0])) { return; }
+        if (isHistory) {
+            APP.history = !state;
+        } else if (!isSf) {
+            APP.online = state;
+        }
+        state = APP.online && !APP.history && state;
         APP.editable = !APP.readOnly && state;
+
         if (!state) {
             APP.$content.addClass('cp-app-drive-readonly');
-            if (!isHistory) {
+            if (!APP.history || !APP.online) {
                 $('#cp-app-drive-connection-state').show();
+            } else {
+                $('#cp-app-drive-connection-state').hide();
             }
             $('[draggable="true"]').attr('draggable', false);
         }
@@ -3661,6 +3674,15 @@ define([
             }
 
             var readOnlyFolder = false;
+
+            // If the shared folder is offline, add the "DISCONNECTED" banner, otherwise
+            // use the normal "editable" behavior (based on drive offline or history mode)
+            if (sfId && manager.folders[sfId].offline) {
+                setEditable(false, false, true);
+            } else {
+                setEditable(true, false, true);
+            }
+
             if (APP.readOnly) {
                 // Read-only drive (team?)
                 $content.prepend($readOnly.clone());
@@ -4140,6 +4162,17 @@ define([
                 data.name = Util.fixFileName(folderName);
                 data.folderName = Util.fixFileName(folderName) + '.zip';
 
+                var uo = manager.user.userObject;
+                if (sfId && manager.folders[sfId]) {
+                    uo = manager.folders[sfId].userObject;
+                }
+                if (uo.getFilesRecursively) {
+                    data.list = uo.getFilesRecursively(folderElement).map(function (el) {
+                        var d = uo.getFileData(el);
+                        return d.channel;
+                    });
+                }
+
                 APP.FM.downloadFolder(data, function (err, obj) {
                     console.log(err, obj);
                     console.log('DONE');

www/common/inner/cache.js

@@ -0,0 +1,33 @@
+define([
+], function () {
+    var S = {};
+
+    S.create = function (sframeChan) {
+        var getBlobCache = function (id, cb) {
+            sframeChan.query('Q_GET_BLOB_CACHE', {id:id}, function (err, data) {
+                var e = err || (data && data.error);
+                if (e) { return void cb(e); }
+                if (!data || typeof(data) !== "object") { return void cb('EINVAL'); }
+                cb(null, data);
+            }, { raw: true });
+        };
+        var setBlobCache = function (id, u8, cb) {
+            sframeChan.query('Q_SET_BLOB_CACHE', {
+                id: id,
+                u8: u8
+            }, function (err, data) {
+                var e = err || (data && data.error) || undefined;
+                cb(e);
+            }, { raw: true });
+        };
+
+
+        return {
+            getBlobCache: getBlobCache,
+            setBlobCache: setBlobCache
+        };
+    };
+
+    return S;
+});
+

www/common/inner/common-mediatag.js

@@ -17,11 +17,17 @@ define([
     var Nacl = window.nacl;
 
     // Configure MediaTags to use our local viewer
+    // This file is loaded by sframe-common so the following config is used in all the inner apps
     if (MediaTag) {
         MediaTag.setDefaultConfig('pdf', {
             viewer: '/common/pdfjs/web/viewer.html'
         });
+        MediaTag.setDefaultConfig('download', {
+            text: Messages.mediatag_saveButton,
+            textDl: Messages.mediatag_loadButton,
+        });
     }
+    MT.MediaTag = MediaTag;
 
     // Cache of the avatars outer html (including <media-tag>)
     var avatars = {};
@@ -68,7 +74,7 @@ define([
             childList: true,
             characterData: false
         });
-        MediaTag($tag[0]).on('error', function (data) {
+        MediaTag($tag[0], {force: true}).on('error', function (data) {
             console.error(data);
         });
     };
@@ -241,7 +247,6 @@ define([
         var locked = false;
         var show = function (_i) {
             if (locked) { return; }
-            locked = true;
             if (_i < 0) { i = 0; }
             else if (_i > tags.length -1) { i = tags.length - 1; }
             else { i = _i; }
@@ -285,7 +290,6 @@ define([
                     if (_key) { key = 'cryptpad:' + Nacl.util.encodeBase64(_key); }
                 }
                 if (!src || !key) {
-                    locked = false;
                     $spinner.hide();
                     return void UI.log(Messages.error);
                 }
@@ -299,13 +303,18 @@ define([
                         locked = false;
                         $spinner.hide();
                         UI.log(Messages.error);
+                    }).on('progress', function () {
+                        $spinner.hide();
+                        locked = true;
+                    }).on('complete', function () {
+                        locked = false;
+                        $spinner.hide();
                     });
                 });
             }
 
             var observer = new MutationObserver(function(mutations) {
                 mutations.forEach(function() {
-                    locked = false;
                     $spinner.hide();
                 });
             });
@@ -377,6 +386,14 @@ define([
                     'tabindex': '-1',
                     'data-icon': "fa-eye",
                 }, Messages.pad_mediatagPreview)),
+                h('li.cp-svg', h('a.cp-app-code-context-openin.dropdown-item', {
+                    'tabindex': '-1',
+                    'data-icon': "fa-external-link",
+                }, Messages.pad_mediatagOpen)),
+                h('li.cp-svg', h('a.cp-app-code-context-share.dropdown-item', {
+                    'tabindex': '-1',
+                    'data-icon': "fa-shhare-alt",
+                }, Messages.pad_mediatagShare)),
                 h('li', h('a.cp-app-code-context-saveindrive.dropdown-item', {
                     'tabindex': '-1',
                     'data-icon': "fa-cloud-upload",
@@ -413,12 +430,29 @@ define([
             }
             else if ($this.hasClass("cp-app-code-context-download")) {
                 var media = Util.find($mt, [0, '_mediaObject']);
+                if (!media) { return void console.error('no media'); }
+                if (!media.complete) { return void UI.warn(Messages.mediatag_notReady); }
                 if (!(media && media._blob)) { return void console.error($mt); }
                 window.saveAs(media._blob.content, media.name);
             }
             else if ($this.hasClass("cp-app-code-context-open")) {
                 $mt.trigger('preview');
             }
+            else if ($this.hasClass("cp-app-code-context-openin")) {
+                var hash = common.getHashFromMediaTag($mt);
+                common.openURL(Hash.hashToHref(hash, 'file'));
+            }
+            else if ($this.hasClass("cp-app-code-context-share")) {
+                var data = {
+                    file: true,
+                    pathname: '/file/',
+                    hashes: {
+                        fileHash: common.getHashFromMediaTag($mt)
+                    },
+                    title: Util.find($mt[0], ['_mediaObject', 'name']) || ''
+                };
+                common.getSframeChannel().event('EV_SHARE_OPEN', data);
+            }
         });
 
         return m;

www/common/inner/properties.js

@@ -19,6 +19,13 @@ define([
         var $d = $('<div>');
         if (!data) { return void cb(void 0, $d); }
 
+        if (data.channel) {
+            $('<label>', { 'for': 'cp-app-prop-id'}).text(Messages.documentID).appendTo($d);
+            $d.append(UI.dialog.selectable(data.channel, {
+                id: 'cp-app-prop-id',
+            }));
+        }
+
         if (data.href) {
             $('<label>', {'for': 'cp-app-prop-link'}).text(Messages.editShare).appendTo($d);
             $d.append(UI.dialog.selectable(data.href, {

www/common/inner/share.js

@@ -345,7 +345,7 @@ define([
 
             localStore.get('hide-alert-shareLinkWarning', function (val) {
                 if (val === '1') { return; }
-                $(shareLinkWarning).show();
+                $(shareLinkWarning).css('display', 'flex');
 
                 $(dismissButton).on('click', function () {
                     localStore.put('hide-alert-shareLinkWarning', '1');

www/common/make-backup.js

@@ -1,17 +1,18 @@
 define([
     'jquery',
-    '/common/cryptget.js',
     '/file/file-crypto.js',
     '/common/common-hash.js',
     '/common/common-util.js',
     '/common/common-interface.js',
     '/common/hyperscript.js',
     '/common/common-feedback.js',
+    '/common/inner/cache.js',
     '/customize/messages.js',
     '/bower_components/nthen/index.js',
     '/bower_components/saferphore/index.js',
     '/bower_components/jszip/dist/jszip.min.js',
-], function ($, Crypt, FileCrypto, Hash, Util, UI, h, Feedback, Messages, nThen, Saferphore, JsZip) {
+], function ($, FileCrypto, Hash, Util, UI, h, Feedback,
+             Cache, Messages, nThen, Saferphore, JsZip) {
     var saveAs = window.saveAs;
 
     var sanitize = function (str) {
@@ -53,9 +54,6 @@ define([
 
     var _downloadFile = function (ctx, fData, cb, updateProgress) {
         var cancelled = false;
-        var cancel = function () {
-            cancelled = true;
-        };
         var href = (fData.href && fData.href.indexOf('#') !== -1) ? fData.href : fData.roHref;
         var parsed = Hash.parsePadUrl(href);
         var hash = parsed.hash;
@@ -63,10 +61,13 @@ define([
         var secret = Hash.getSecrets('file', hash, fData.password);
         var src = (ctx.fileHost || '') + Hash.getBlobPathFromHex(secret.channel);
         var key = secret.keys && secret.keys.cryptKey;
-        Util.fetch(src, function (err, u8) {
+
+        var fetchObj, decryptObj;
+
+        fetchObj = Util.fetch(src, function (err, u8) {
             if (cancelled) { return; }
             if (err) { return void cb('E404'); }
-            FileCrypto.decrypt(u8, key, function (err, res) {
+            decryptObj = FileCrypto.decrypt(u8, key, function (err, res) {
                 if (cancelled) { return; }
                 if (err) { return void cb(err); }
                 if (!res.content) { return void cb('EEMPTY'); }
@@ -78,8 +79,25 @@ define([
                     content: res.content,
                     download: dl
                 });
-            }, updateProgress && updateProgress.progress2);
+            }, function (data) {
-        }, updateProgress && updateProgress.progress);
+                if (cancelled) { return; }
+                if (updateProgress && updateProgress.progress2) {
+                    updateProgress.progress2(data);
+                }
+            });
+        }, function (data) {
+            if (cancelled) { return; }
+            if (updateProgress && updateProgress.progress) {
+                updateProgress.progress(data);
+            }
+        }, ctx.cache);
+
+        var cancel = function () {
+            cancelled = true;
+            if (fetchObj && fetchObj.cancel) { fetchObj.cancel(); }
+            if (decryptObj && decryptObj.cancel) { decryptObj.cancel(); }
+        };
+
         return {
             cancel: cancel
         };
@@ -162,10 +180,10 @@ define([
                 if (ctx.stop) { return; }
                 if (to) { clearTimeout(to); }
                 //setTimeout(g, 2000);
-                g();
-                w();
                 ctx.done++;
                 ctx.updateProgress('download', {max: ctx.max, current: ctx.done});
+                g();
+                w();
             };
 
             var error = function (err) {
@@ -274,7 +292,7 @@ define([
     };
 
     // Main function. Create the empty zip and fill it starting from drive.root
-    var create = function (data, getPad, fileHost, cb, progress) {
+    var create = function (data, getPad, fileHost, cb, progress, cache) {
         if (!data || !data.uo || !data.uo.drive) { return void cb('EEMPTY'); }
         var sem = Saferphore.create(5);
         var ctx = {
@@ -288,7 +306,8 @@ define([
             sem: sem,
             updateProgress: progress,
             max: 0,
-            done: 0
+            done: 0,
+            cache: cache
         };
         var filesData = data.sharedFolderId && ctx.sf[data.sharedFolderId] ? ctx.sf[data.sharedFolderId].filesData : ctx.data.filesData;
         progress('reading', -1);
@@ -312,13 +331,14 @@ define([
             delete ctx.zip;
         };
         return {
-            stop: stop
+            stop: stop,
+            cancel: stop
         };
     };
 
 
     var _downloadFolder = function (ctx, data, cb, updateProgress) {
-        create(data, ctx.get, ctx.fileHost, function (blob, errors) {
+        return create(data, ctx.get, ctx.fileHost, function (blob, errors) {
             if (errors && errors.length) { console.error(errors); } // TODO show user errors
             var dl = function () {
                 saveAs(blob, data.folderName);
@@ -332,10 +352,13 @@ define([
                 if (typeof progress.current !== "number") { return; }
                 updateProgress.folderProgress(progress.current / progress.max);
             }
+            else if (state === "compressing") {
+                updateProgress.folderProgress(2);
+            }
             else if (state === "done") {
-                updateProgress.folderProgress(1);
+                updateProgress.folderProgress(3);
             }
-        });
+        }, ctx.cache);
     };
 
     var createExportUI = function (origin) {

www/common/media-tag.js

@@ -1,8 +1,6 @@
-(function(name, definition) {
+(function (window) {
-    if (typeof module !== 'undefined') { module.exports = definition(); }
+var factory = function () {
-    else if (typeof define === 'function' && typeof define.amd === 'object') { define(definition); }
+    var Promise = window.Promise;
-    else  { this[name] = definition(); }
-}('MediaTag', function() {
     var cache;
     var cypherChunkLength = 131088;
 
@@ -50,6 +48,7 @@
             'image/jpeg',
             'image/jpg',
             'image/gif',
+            'audio/mpeg',
             'audio/mp3',
             'audio/ogg',
             'audio/wav',
@@ -63,7 +62,8 @@
         ],
         pdf: {},
         download: {
-            text: "Download"
+            text: "Save",
+            textDl: "Load attachment"
         },
         Plugins: {
             /**
@@ -114,8 +114,8 @@
             },
             download: function (metadata, url, content, cfg, cb) {
                 var btn = document.createElement('button');
-                btn.setAttribute('class', 'btn btn-success');
+                btn.setAttribute('class', 'btn btn-default');
-                btn.innerHTML = cfg.download.text + '<br>' +
+                btn.innerHTML = '<i class="fa fa-save"></i>' + cfg.download.text + '<br>' +
                                 (metadata.name ? '<b>' + fixHTML(metadata.name) + '</b>' : '');
                 btn.addEventListener('click', function () {
                     saveFile(content, url, metadata.name);
@@ -125,28 +125,186 @@
         }
     };
 
+    var makeProgressBar = function (cfg, mediaObject) {
+        if (mediaObject.bar) { return; }
+        mediaObject.bar = true;
+        var style = (function(){/*
+.mediatag-progress-container {
+    position: relative;
+    border: 1px solid #0087FF;
+    background: white;
+    height: 25px;
+    display: inline-flex;
+    width: 200px;
+    align-items: center;
+    justify-content: center;
+    box-sizing: border-box;
+    vertical-align: top;
+}
+.mediatag-progress-bar {
+    position: absolute;
+    left: 0;
+    top: 0;
+    bottom: 0;
+    background: #0087FF;
+    width: 0%;
+}
+.mediatag-progress-text {
+    font-size: 14px;
+    height: 25px;
+    width: 50px;
+    margin-left: 5px;
+    line-height: 25px;
+    vertical-align: top;
+    display: inline-block;
+    color: #3F4141;
+    font-weight: bold;
+}
+*/}).toString().slice(14, -3);
+        var container = document.createElement('div');
+        container.classList.add('mediatag-progress-container');
+        var bar = document.createElement('div');
+        bar.classList.add('mediatag-progress-bar');
+        container.appendChild(bar);
+
+        var text = document.createElement('span');
+        text.classList.add('mediatag-progress-text');
+        text.innerText = '0%';
+
+        mediaObject.on('progress', function (obj) {
+            var percent = obj.progress;
+            text.innerText = (Math.round(percent*10))/10+'%';
+            bar.setAttribute('style', 'width:'+percent+'%;');
+        });
+
+        mediaObject.tag.innerHTML = '<style>'+style+'</style>';
+        mediaObject.tag.appendChild(container);
+        mediaObject.tag.appendChild(text);
+    };
+    var makeDownloadButton = function (cfg, mediaObject, size, cb) {
+        var metadata = cfg.metadata || {};
+        var i = '<i class="fa fa-paperclip"></i>';
+        var name = metadata.name ? '<span class="mediatag-download-name">'+ i +'<b>'+
+                                    fixHTML(metadata.name)+'</b></span>' : '';
+        var btn = document.createElement('button');
+        btn.setAttribute('class', 'btn btn-default mediatag-download-btn');
+        btn.innerHTML = name + '<span>' + (name ? '' : i) +
+                cfg.download.textDl + ' <b>(' + size  + 'MB)</b></span>';
+        btn.addEventListener('click', function () {
+            makeProgressBar(cfg, mediaObject);
+            var a = (cfg.body || document).querySelectorAll('media-tag[src="'+mediaObject.tag.getAttribute('src')+'"] button.mediatag-download-btn');
+            for(var i = 0; i < a.length; i++) {
+                if (a[i] !== btn) { a[i].click(); }
+            }
+            cb();
+        });
+        mediaObject.tag.innerHTML = '';
+        mediaObject.tag.appendChild(btn);
+    };
+
+    var getCacheKey = function (src) {
+        var _src = src.replace(/(\/)*$/, ''); // Remove trailing slashes
+        var idx = _src.lastIndexOf('/');
+        var cacheKey = _src.slice(idx+1);
+        if (!/^[a-f0-9]{48}$/.test(cacheKey)) { cacheKey = undefined; }
+        return cacheKey;
+    };
+
+    var getBlobCache = function (id, cb) {
+        if (!config.Cache || typeof(config.Cache.getBlobCache) !== "function") {
+            return void cb('EINVAL');
+        }
+        config.Cache.getBlobCache(id, cb);
+    };
+    var setBlobCache = function (id, u8, cb) {
+        if (!config.Cache || typeof(config.Cache.setBlobCache) !== "function") {
+            return void cb('EINVAL');
+        }
+        config.Cache.setBlobCache(id, u8, cb);
+    };
+
+    var getFileSize = function (src, _cb) {
+        var cb = function (e, res) {
+            _cb(e, res);
+            cb = function () {};
+        };
+
+        var cacheKey = getCacheKey(src);
+
+        var check = function () {
+            var xhr = new XMLHttpRequest();
+            xhr.open("HEAD", src);
+            xhr.onerror = function () { return void cb("XHR_ERROR"); };
+            xhr.onreadystatechange = function() {
+                if (this.readyState === this.DONE) {
+                    cb(null, Number(xhr.getResponseHeader("Content-Length")));
+                }
+            };
+            xhr.onload = function () {
+                if (/^4/.test('' + this.status)) { return void cb("XHR_ERROR " + this.status); }
+            };
+            xhr.send();
+        };
+
+        if (!cacheKey) { return void check(); }
+
+        getBlobCache(cacheKey, function (err, u8) {
+            if (err || !u8) { return void check(); }
+            cb(null, 0);
+        });
+    };
 
     // Download a blob from href
-    var download = function (src, _cb) {
+    var download = function (src, _cb, progressCb) {
         var cb = function (e, res) {
             _cb(e, res);
             cb = function () {};
         };
 
-        var xhr = new XMLHttpRequest();
+        var cacheKey = getCacheKey(src);
-        xhr.open('GET', src, true);
+
-        xhr.responseType = 'arraybuffer';
+        var fetch = function () {
+            var xhr = new XMLHttpRequest();
+            xhr.open('GET', src, true);
+            xhr.responseType = 'arraybuffer';
 
-        xhr.onerror = function () { return void cb("XHR_ERROR"); };
+            var progress = function (offset) {
-        xhr.onload = function () {
+                progressCb(offset * 100);
-            // Error?
+            };
-            if (/^4/.test('' + this.status)) { return void cb("XHR_ERROR " + this.status); }
+            xhr.addEventListener("progress", function (evt) {
+                if (evt.lengthComputable) {
+                    var percentComplete = evt.loaded / evt.total;
+                    progress(percentComplete);
+                }
+            }, false);
+
+            xhr.onerror = function () { return void cb("XHR_ERROR"); };
+            xhr.onload = function () {
+                // Error?
+                if (/^4/.test('' + this.status)) { return void cb("XHR_ERROR " + this.status); }
+
+                var arrayBuffer = xhr.response;
+                if (arrayBuffer) {
+                    var u8 = new Uint8Array(arrayBuffer);
+                    if (cacheKey) {
+                        return void setBlobCache(cacheKey, u8, function () {
+                            cb(null, u8);
+                        });
+                    }
+                    cb(null, u8);
+                }
+            };
 
-            var arrayBuffer = xhr.response;
+            xhr.send(null);
-            if (arrayBuffer) { cb(null, new Uint8Array(arrayBuffer)); }
         };
 
-        xhr.send(null);
+        if (!cacheKey) { return void fetch(); }
+
+        getBlobCache(cacheKey, function (err, u8) {
+            if (err || !u8) { return void fetch(); }
+            cb(null, u8);
+        });
+
     };
 
     // Decryption tools
@@ -192,6 +350,92 @@
         }
     };
 
+    // The metadata size can go up to 65535 (16 bits - 2 bytes)
+    // The first 8 bits are stored in A[0]
+    // The last 8 bits are stored in A[0]
+    var uint8ArrayJoin = function (AA) {
+        var l = 0;
+        var i = 0;
+        for (; i < AA.length; i++) { l += AA[i].length; }
+        var C = new Uint8Array(l);
+
+        i = 0;
+        for (var offset = 0; i < AA.length; i++) {
+            C.set(AA[i], offset);
+            offset += AA[i].length;
+        }
+        return C;
+    };
+    var fetchMetadata = function (src, _cb) {
+        var cb = function (e, res) {
+            _cb(e, res);
+            cb = function () {};
+        };
+
+        var cacheKey = getCacheKey(src);
+
+        var fetch = function () {
+            var xhr = new XMLHttpRequest();
+            xhr.open('GET', src, true);
+            xhr.setRequestHeader('Range', 'bytes=0-1');
+            xhr.responseType = 'arraybuffer';
+
+            xhr.onerror = function () { return void cb("XHR_ERROR"); };
+            xhr.onload = function () {
+                // Error?
+                if (/^4/.test('' + this.status)) { return void cb("XHR_ERROR " + this.status); }
+                var res = new Uint8Array(xhr.response);
+                var size = Decrypt.decodePrefix(res);
+                var xhr2 = new XMLHttpRequest();
+
+                xhr2.open("GET", src, true);
+                xhr2.setRequestHeader('Range', 'bytes=2-' + (size + 2));
+                xhr2.responseType = 'arraybuffer';
+                xhr2.onload = function () {
+                    if (/^4/.test('' + this.status)) { return void cb("XHR_ERROR " + this.status); }
+                    var res2 = new Uint8Array(xhr2.response);
+                    var all = uint8ArrayJoin([res, res2]);
+                    cb(void 0, all);
+                };
+                xhr2.send(null);
+            };
+
+            xhr.send(null);
+        };
+
+        if (!cacheKey) { return void fetch(); }
+
+        getBlobCache(cacheKey, function (err, u8) {
+            if (err || !u8) { return void fetch(); }
+            var size = Decrypt.decodePrefix(u8.subarray(0,2));
+            cb(null, u8.subarray(0, size+2));
+        });
+    };
+    var decryptMetadata = function (u8, key) {
+        var prefix = u8.subarray(0, 2);
+        var metadataLength = Decrypt.decodePrefix(prefix);
+
+        var metaBox = new Uint8Array(u8.subarray(2, 2 + metadataLength));
+        var metaChunk = window.nacl.secretbox.open(metaBox, Decrypt.createNonce(), key);
+
+        try {
+            return JSON.parse(window.nacl.util.encodeUTF8(metaChunk));
+        }
+        catch (e) { return null; }
+    };
+    var fetchDecryptedMetadata = function (src, strKey, cb) {
+        if (typeof(src) !== 'string') {
+            return window.setTimeout(function () {
+                cb('NO_SOURCE');
+            });
+        }
+        fetchMetadata(src, function (e, buffer) {
+            if (e) { return cb(e); }
+            var key = Decrypt.getKeyFromStr(strKey);
+            cb(void 0, decryptMetadata(buffer, key));
+        });
+    };
+
     // Decrypts a Uint8Array with the given key.
     var decrypt = function (u8, strKey, done, progressCb) {
         var Nacl = window.nacl;
@@ -372,6 +616,7 @@
         var handlers = cfg.handlers || {
             'progress': [],
             'complete': [],
+            'metadata': [],
             'error': []
         };
 
@@ -422,6 +667,7 @@
 
         // End media-tag rendering: display the tag and emit the event
         var end = function (decrypted) {
+            mediaObject.complete = true;
             process(mediaObject, decrypted, cfg, function (err) {
                 if (err) { return void emit('error', err); }
                 mediaObject._blob = decrypted;
@@ -429,32 +675,78 @@
             });
         };
 
-        // If we have the blob in our cache, don't download & decrypt it again, just display
+        var error = function (err) {
-        if (cache[uid]) {
+            mediaObject.tag.innerHTML = '<img style="width: 100px; height: 100px;" src="/images/broken.png">';
-            end(cache[uid]);
+            emit('error', err);
-            return mediaObject;
+        };
-        }
 
-        // Download the encrypted blob
+        var getCache = function () {
-        download(src, function (err, u8Encrypted) {
+            var c = cache[uid];
-            if (err) {
+            if (!c || !c.promise || !c.mt) { return; }
-                if (err === "XHR_ERROR 404") {
+            return c;
-                    mediaObject.tag.innerHTML = '<img style="width: 100px; height: 100px;" src="/images/broken.png">';
+        };
-                }
+
-                return void emit('error', err);
+        var dl = function () {
+            // Download the encrypted blob
+            cache[uid] = getCache() || {
+                promise: new Promise(function (resolve, reject) {
+                    download(src, function (err, u8Encrypted) {
+                        if (err) {
+                            return void reject(err);
+                        }
+                        // Decrypt the blob
+                        decrypt(u8Encrypted, strKey, function (errDecryption, u8Decrypted) {
+                            if (errDecryption) {
+                                return void reject(errDecryption);
+                            }
+                            emit('metadata', u8Decrypted.metadata);
+                            resolve(u8Decrypted);
+                        }, function (progress) {
+                            emit('progress', {
+                                progress: 50+0.5*progress
+                            });
+                        });
+                    }, function (progress) {
+                        emit('progress', {
+                            progress: 0.5*progress
+                        });
+                    });
+                }),
+                mt: mediaObject
+            };
+            if (cache[uid].mt !== mediaObject) {
+                // Add progress for other instances of this tag
+                cache[uid].mt.on('progress', function (obj) {
+                    if (!mediaObject.bar && !cfg.force) { makeProgressBar(cfg, mediaObject); }
+                    emit('progress', {
+                        progress: obj.progress
+                    });
+                });
             }
-            // Decrypt the blob
+            cache[uid].promise.then(function (u8) {
-            decrypt(u8Encrypted, strKey, function (errDecryption, u8Decrypted) {
+                end(u8);
-                if (errDecryption) {
+            }, function (err) {
-                    return void emit('error', errDecryption);
+                error(err);
+            });
+        };
+
+        if (cfg.force) { dl(); return mediaObject; }
+
+        var maxSize = typeof(config.maxDownloadSize) === "number" ? config.maxDownloadSize
+                                : (5 * 1024 * 1024);
+        fetchDecryptedMetadata(src, strKey, function (err, md) {
+            if (err) { return void error(err); }
+            cfg.metadata = md;
+            emit('metadata', md);
+            getFileSize(src, function (err, size) {
+                // If the size is smaller than the autodownload limit, load the blob.
+                // If the blob is already loaded or being loaded, don't show the button.
+                if (!size || size <  maxSize || getCache()) {
+                    makeProgressBar(cfg, mediaObject);
+                    return void dl();
                 }
-                // Cache and display the decrypted blob
+                var sizeMb = Math.round(10 * size / 1024 / 1024) / 10;
-                cache[uid] = u8Decrypted;
+                makeDownloadButton(cfg, mediaObject, sizeMb, dl);
-                end(u8Decrypted);
-            }, function (progress) {
-                emit('progress', {
-                    progress: progress
-                });
             });
         });
 
@@ -468,5 +760,20 @@
         config[key] = value;
     };
 
+    init.fetchDecryptedMetadata = fetchDecryptedMetadata;
+
     return init;
-}));
+};
+
+    if (typeof(module) !== 'undefined' && module.exports) {
+        module.exports = factory();
+    } else if ((typeof(define) !== 'undefined' && define !== null) && (define.amd !== null)) {
+        define([
+            '/bower_components/es6-promise/es6-promise.min.js'
+        ], function () {
+            return factory();
+        });
+    } else {
+        // unsupported initialization
+    }
+}(typeof(window) !== 'undefined'? window : {}));

www/common/notify.js

@@ -20,6 +20,8 @@ define(['/api/config'], function (ApiConfig) {
 
     var getPermission = Module.getPermission = function (f) {
         f = f || function () {};
+        // "Notification.requestPermission is not a function" on Firefox 68.11.0esr
+        if (!Notification || typeof(Notification.requestPermission) !== 'function') { return void f(false); }
         Notification.requestPermission(function (permission) {
             if (permission === "granted") { f(true); }
             else { f(false); }

www/common/onlyoffice/inner.js

@@ -99,6 +99,9 @@ define([
         var sessionId = Hash.createChannelId();
         var cpNfInner;
 
+        var evOnPatch = Util.mkEvent();
+        var evOnSync = Util.mkEvent();
+
         // This structure is used for caching media data and blob urls for each media cryptpad url
         var mediasData = {};
 
@@ -261,13 +264,17 @@ define([
                 });
             },
             sendMsg: function (msg, cp, cb) {
+                evOnPatch.fire();
                 rtChannel.sendCmd({
                     cmd: 'SEND_MESSAGE',
                     data: {
                         msg: msg,
                         isCp: cp
                     }
-                }, cb);
+                }, function (err, h) {
+                    if (!err) { evOnSync.fire(); }
+                    cb(err, h);
+                });
             },
         };
 
@@ -775,6 +782,18 @@ define([
                     view: false
                 };
             });
+            // Add an history keeper user to show that we're never alone
+            var hkId = Util.createRandomInteger();
+            p.push({
+                id: hkId,
+                idOriginal: String(hkId),
+                username: "History",
+                indexUser: i,
+                connectionId: Hash.createChannelId(),
+                isCloseCoAuthoring:false,
+                view: false
+            });
+            i++;
             if (!myUniqueOOId) { myUniqueOOId = String(myOOId) + i; }
             p.push({
                 id: myUniqueOOId,
@@ -1221,6 +1240,7 @@ define([
                         }
                     },
                     "onDocumentReady": function () {
+                        evOnSync.fire();
                         var onMigrateRdy = Util.mkEvent();
                         onMigrateRdy.reg(function () {
                             var div = h('div.cp-oo-x2tXls', [
@@ -1301,6 +1321,8 @@ define([
                             return;
                         }
 
+                        APP.onLocal(); // Add our data to the userlist
+
                         if (APP.history) {
                             try {
                                 getEditor().asc_setRestriction(true);
@@ -1310,6 +1332,16 @@ define([
                         if (APP.migrate && !readOnly) {
                             onMigrateRdy.fire();
                         }
+
+                        // Check if history can/should be trimmed
+                        var cp = getLastCp();
+                        if (cp && cp.file && cp.hash) {
+                            var channels = [{
+                                channel: content.channel,
+                                lastKnownHash: cp.hash
+                            }];
+                            common.checkTrimHistory(channels);
+                        }
                     }
                 }
             };
@@ -1355,6 +1387,10 @@ define([
                 });
             };
 
+            APP.openURL = function (url) {
+                common.openUnsafeURL(url);
+            };
+
             APP.loadingImage = 0;
             APP.getImageURL = function(name, callback) {
                 var mediasSources = getMediasSources();
@@ -1409,7 +1445,7 @@ define([
                         console.error(e);
                         callback("");
                     }
-                });
+                }, void 0, common.getCache());
             };
 
             APP.docEditor = new window.DocsAPI.DocEditor("cp-app-oo-placeholder-a", APP.ooconfig);
@@ -1949,6 +1985,10 @@ define([
                 metadataMgr: metadataMgr,
                 readOnly: readOnly,
                 realtime: info.realtime,
+                spinner: {
+                    onPatch: evOnPatch,
+                    onSync: evOnSync
+                },
                 sfCommon: common,
                 $container: $bar,
                 $contentContainer: $('#cp-app-oo-container')
@@ -2166,6 +2206,12 @@ define([
                     url: newLatest.file
                 }, function () { });
                 newDoc = !content.hashes || Object.keys(content.hashes).length === 0;
+            } else if (!privateData.isNewFile) {
+                // This is an empty doc but not a new file: error
+                UI.errorLoadingScreen(Messages.unableToDisplay, false, function () {
+                    common.gotoURL('');
+                });
+                throw new Error("Empty chainpad for a non-empty doc");
             } else {
                 Title.updateTitle(Title.defaultTitle);
             }

www/common/onlyoffice/main.js

@@ -12,7 +12,7 @@ define([
     nThen(function (waitFor) {
         DomReady.onReady(waitFor());
     }).nThen(function (waitFor) {
-        var obj = SFCommonO.initIframe(waitFor, true, true);
+        var obj = SFCommonO.initIframe(waitFor, true);
         href = obj.href;
         hash = obj.hash;
         var parsed = Hash.parsePadUrl(href);

www/common/outer/async-store.js

@@ -10,6 +10,7 @@ define([
     '/common/common-realtime.js',
     '/common/common-messaging.js',
     '/common/pinpad.js',
+    '/common/outer/cache-store.js',
     '/common/outer/sharedfolder.js',
     '/common/outer/cursor.js',
     '/common/outer/onlyoffice.js',
@@ -28,7 +29,7 @@ define([
     '/bower_components/nthen/index.js',
     '/bower_components/saferphore/index.js',
 ], function (Sortify, UserObject, ProxyManager, Migrate, Hash, Util, Constants, Feedback,
-             Realtime, Messaging, Pinpad,
+             Realtime, Messaging, Pinpad, Cache,
              SF, Cursor, OnlyOffice, Mailbox, Profile, Team, Messenger, History,
              NetConfig, AppConfig,
              Crypto, ChainPad, CpNetflux, Listmap, nThen, Saferphore) {
@@ -120,10 +121,13 @@ define([
         Store.getSharedFolder = function (clientId, data, cb) {
             var s = getStore(data.teamId);
             var id = data.id;
+            var proxy;
             if (!s || !s.manager) { return void cb({ error: 'ENOTFOUND' }); }
             if (s.manager.folders[id]) {
+                proxy = Util.clone(s.manager.folders[id].proxy);
+                proxy.offline = Boolean(s.manager.folders[id].offline);
                 // If it is loaded, return the shared folder proxy
-                return void cb(s.manager.folders[id].proxy);
+                return void cb(proxy);
             } else {
                 // Otherwise, check if we know this shared folder
                 var shared = Util.find(s.proxy, ['drive', UserObject.SHARED_FOLDERS]) || {};
@@ -327,6 +331,7 @@ define([
             if (!s.rpc) { return void cb({error: 'RPC_NOT_READY'}); }
 
             s.rpc.removeOwnedChannel(channel, function (err) {
+                if (!err) { Cache.clearChannel(channel); }
                 cb({error:err});
             });
         };
@@ -459,6 +464,7 @@ define([
             store.anon_rpc.send("GET_FILE_SIZE", channelId, function (e, response) {
                 if (e) { return void cb({error: e}); }
                 if (response && response.length && typeof(response[0]) === 'number') {
+                    if (response[0] === 0) { Cache.clearChannel(channelId); }
                     return void cb({size: response[0]});
                 } else {
                     cb({error: 'INVALID_RESPONSE'});
@@ -472,6 +478,7 @@ define([
             store.anon_rpc.send("IS_NEW_CHANNEL", channelId, function (e, response) {
                 if (e) { return void cb({error: e}); }
                 if (response && response.length && typeof(response[0]) === 'boolean') {
+                    if (response[0]) { Cache.clearChannel(channelId); }
                     return void cb({
                         isNew: response[0]
                     });
@@ -1133,7 +1140,7 @@ define([
             var ownedByMe = Array.isArray(owners) && owners.indexOf(edPublic) !== -1;
 
             // Add the pad if it does not exist in our drive
-            if (!contains || (ownedByMe && !inMyDrive)) {
+            if (!contains) { // || (ownedByMe && !inMyDrive)) {
                 var autoStore = Util.find(store.proxy, ['settings', 'general', 'autostore']);
                 if (autoStore !== 1 && !data.forceSave && !data.path && !ownedByMe) {
                     // send event to inner to display the corner popup
@@ -1326,13 +1333,16 @@ define([
 
             store.proxy.friends_pending = store.proxy.friends_pending || {};
 
-            var twoDaysAgo = +new Date() - (2 * 24 * 3600 * 1000);
+            var p = store.proxy.friends_pending[data.curvePublic];
-            if (store.proxy.friends_pending[data.curvePublic] &&
+            if (p) {
-                    store.proxy.friends_pending[data.curvePublic] > twoDaysAgo) {
+                return void cb({error: 'ALREADY_SENT'});
-                return void cb({error: 'TIMEOUT'});
             }
 
-            store.proxy.friends_pending[data.curvePublic] = +new Date();
+            store.proxy.friends_pending[data.curvePublic] = {
+                time: +new Date(),
+                channel: data.notifications,
+                curvePublic: data.curvePublic
+            };
             broadcast([], "UPDATE_METADATA");
 
             store.mailbox.sendTo('FRIEND_REQUEST', {
@@ -1344,6 +1354,37 @@ define([
                 cb(obj);
             });
         };
+        Store.cancelFriendRequest = function (data, cb) {
+            if (!data.curvePublic || !data.notifications) {
+                return void cb({error: 'EINVAL'});
+            }
+
+            var proxy = store.proxy;
+            var f = Messaging.getFriend(proxy, data.curvePublic);
+
+            if (f) {
+                // Already friend
+                console.error("You can't cancel an accepted friend request");
+                return void cb({error: 'ALREADY_FRIEND'});
+            }
+
+            var pending = Util.find(store, ['proxy', 'friends_pending']) || {};
+            if (!pending) { return void cb(); }
+
+            store.mailbox.sendTo('CANCEL_FRIEND_REQUEST', {
+                user: Messaging.createData(store.proxy)
+            }, {
+                channel: data.notifications,
+                curvePublic: data.curvePublic
+            }, function (obj) {
+                if (obj && obj.error) { return void cb(obj); }
+                delete store.proxy.friends_pending[data.curvePublic];
+                broadcast([], "UPDATE_METADATA");
+                onSync(null, function () {
+                    cb(obj);
+                });
+            });
+        };
 
         Store.anonGetPreviewContent = function (clientId, data, cb) {
             Team.anonGetPreviewContent({
@@ -1590,13 +1631,20 @@ define([
                 Store.leavePad(null, data, function () {});
             };
             var conf = {
+                Cache: Cache, // XXX re-enable cache usage
+                onCacheStart: function () {
+                    postMessage(clientId, "PAD_CACHE");
+                },
+                onCacheReady: function () {
+                    postMessage(clientId, "PAD_CACHE_READY");
+                },
                 onReady: function (pad) {
                     var padData = pad.metadata || {};
                     channel.data = padData;
                     if (padData && padData.validateKey && store.messenger) {
                         store.messenger.storeValidateKey(data.channel, padData.validateKey);
                     }
-                    postMessage(clientId, "PAD_READY");
+                    postMessage(clientId, "PAD_READY", pad.noCache);
                 },
                 onMessage: function (m, user, validateKey, isCp, hash) {
                     channel.lastHash = hash;
@@ -1727,6 +1775,14 @@ define([
             channel.sendMessage(msg, clientId, cb);
         };
 
+        Store.corruptedCache = function (clientId, channel) {
+            var chan = channels[channel];
+            if (!chan || !chan.cpNf) { return; }
+            Cache.clearChannel(channel);
+            if (!chan.cpNf.resetCache) { return; }
+            chan.cpNf.resetCache();
+        };
+
         // Unpin and pin the new channel in all team when changing a pad password
         Store.changePadPasswordPin = function (clientId, data, cb) {
             var oldChannel = data.oldChannel;
@@ -2224,6 +2280,9 @@ define([
             try {
                 store.onlyoffice.leavePad(chanId);
             } catch (e) { console.error(e); }
+            try {
+                Cache.leaveChannel(chanId);
+            } catch (e) { console.error(e); }
 
             if (!Store.channels[chanId]) { return; }
 
@@ -2429,18 +2488,6 @@ define([
             });
         };
 
-        var cleanFriendRequests = function () {
-            try {
-                if (!store.proxy.friends_pending) { return; }
-                var twoDaysAgo = +new Date() - (2 * 24 * 3600 * 1000);
-                Object.keys(store.proxy.friends_pending).forEach(function (curve) {
-                    if (store.proxy.friends_pending[curve] < twoDaysAgo) {
-                        delete store.proxy.friends_pending[curve];
-                    }
-                });
-            } catch (e) {}
-        };
-
         //////////////////////////////////////////////////////////////////
         /////////////////////// Init /////////////////////////////////////
         //////////////////////////////////////////////////////////////////
@@ -2524,7 +2571,6 @@ define([
                 loadUniversal(Profile, 'profile', waitFor);
                 loadUniversal(Team, 'team', waitFor, clientId);
                 loadUniversal(History, 'history', waitFor);
-                cleanFriendRequests();
             }).nThen(function () {
                 var requestLogin = function () {
                     broadcast([], "REQUEST_LOGIN");
@@ -2640,6 +2686,7 @@ define([
                 readOnly: false,
                 validateKey: secret.keys.validateKey || undefined,
                 crypto: Crypto.createEncryptor(secret.keys),
+                Cache: Cache, // XXX re-enable cache usage
                 userName: 'fs',
                 logLevel: 1,
                 ChainPad: ChainPad,

www/common/outer/cache-store.js

@@ -0,0 +1,147 @@
+define([
+    '/common/common-util.js',
+    '/bower_components/localforage/dist/localforage.min.js',
+], function (Util, localForage) {
+    var S = {};
+    var onReady = Util.mkEvent(true);
+
+    // Check if indexedDB is allowed
+    var allowed = false;
+    try {
+        var request = window.indexedDB.open('test_db', 1);
+        request.onsuccess = function () {
+            allowed = true;
+            onReady.fire();
+        };
+        request.onerror = function () {
+            onReady.fire();
+        };
+    } catch (e) {
+        onReady.fire();
+    }
+
+    var cache = localForage.createInstance({
+        driver: localForage.INDEXEDDB,
+        name: "cp_cache"
+    });
+
+    S.getBlobCache = function (id, cb) {
+        cb = Util.once(Util.mkAsync(cb || function () {}));
+
+        onReady.reg(function () {
+            if (!allowed) { return void cb('NOCACHE'); }
+            cache.getItem(id, function (err, obj) {
+                if (err || !obj || !obj.c) {
+                    return void cb(err || 'EINVAL');
+                }
+                cb(null, obj.c);
+                obj.t = +new Date();
+                cache.setItem(id, obj);
+            });
+        });
+    };
+    S.setBlobCache = function (id, u8, cb) {
+        cb = Util.once(Util.mkAsync(cb || function () {}));
+
+        onReady.reg(function () {
+            if (!allowed) { return void cb('NOCACHE'); }
+            if (!u8) { return void cb('EINVAL'); }
+            cache.setItem(id, {
+                c: u8,
+                t: (+new Date()) // 't' represent the "lastAccess" of this cache (get or set)
+            }, function (err) {
+                cb(err);
+            });
+        });
+    };
+
+    // id: channel ID or blob ID
+    // returns array of messages
+    S.getChannelCache = function (id, cb) {
+        cb = Util.once(Util.mkAsync(cb || function () {}));
+
+        onReady.reg(function () {
+            if (!allowed) { return void cb('NOCACHE'); }
+            cache.getItem(id, function (err, obj) {
+                if (err || !obj || !Array.isArray(obj.c)) {
+                    return void cb(err || 'EINVAL');
+                }
+                cb(null, obj);
+                obj.t = +new Date();
+                cache.setItem(id, obj);
+            });
+        });
+    };
+
+    // Keep the last two checkpoint + any checkpoint that may exist in the last 100 messages
+    // FIXME: duplicate system with sliceCpIndex from lib/hk-util.js
+    var checkCheckpoints = function (array) {
+        if (!Array.isArray(array)) { return; }
+        // Keep the last 100 messages
+        if (array.length > 100) {
+            array.splice(0, array.length - 100);
+        }
+        // Remove every message before the first checkpoint
+        var firstCpIdx;
+        array.some(function (el, i) {
+            if (!el.isCheckpoint) { return; }
+            firstCpIdx = i;
+            return true;
+        });
+        array.splice(0, firstCpIdx);
+    };
+
+    var t = {};
+    S.storeCache = function (id, validateKey, val, onError) {
+        onError = Util.once(Util.mkAsync(onError || function () {}));
+
+        onReady.reg(function () {
+
+            // Make a throttle or use the existing one to avoid calling
+            // storeCache with the same array multiple times
+            t[id] = t[id] || Util.throttle(function (validateKey, val, onError) {
+                if (!allowed) { return void onError('NOCACHE'); }
+                if (!Array.isArray(val) || !validateKey) { return void onError('EINVAL'); }
+                checkCheckpoints(val);
+                cache.setItem(id, {
+                    k: validateKey,
+                    c: val,
+                    t: (+new Date()) // 't' represent the "lastAccess" of this cache (get or set)
+                }, function (err) {
+                    if (err) { onError(err); }
+                });
+
+            }, 50);
+            t[id](validateKey, val, onError);
+
+        });
+    };
+
+    S.leaveChannel = function (id) {
+        delete t[id];
+    };
+
+    S.clearChannel = function (id, cb) {
+        cb = Util.once(Util.mkAsync(cb || function () {}));
+
+        onReady.reg(function () {
+            if (!allowed) { return void cb('NOCACHE'); }
+            cache.removeItem(id, function () {
+                cb();
+            });
+        });
+    };
+
+    S.clear = function (cb) {
+        cb = Util.once(Util.mkAsync(cb || function () {}));
+
+        onReady.reg(function () {
+            if (!allowed) { return void cb('NOCACHE'); }
+            cache.clear(cb);
+        });
+    };
+
+    self.CryptPad_clearIndexedDB = S.clear;
+
+    return S;
+});

www/common/outer/local-store.js

@@ -1,9 +1,11 @@
 define([
     '/common/common-constants.js',
     '/common/common-hash.js',
+    '/common/outer/cache-store.js',
     '/bower_components/localforage/dist/localforage.min.js',
     '/customize/application_config.js',
-], function (Constants, Hash, localForage, AppConfig) {
+    '/common/common-util.js',
+], function (Constants, Hash, Cache, localForage, AppConfig, Util) {
     var LocalStore = {};
 
     LocalStore.setThumbnail = function (key, value, cb) {
@@ -119,7 +121,14 @@ define([
             return void AppConfig.customizeLogout(cb);
         }
 
-        if (cb) { cb(); }
+        cb = Util.once(cb || function () {});
+
+        try {
+            Cache.clear(cb);
+        } catch (e) {
+            console.error(e);
+            cb();
+        }
     };
     var loginHandlers = [];
     LocalStore.loginReload = function () {

www/common/outer/mailbox-handlers.js

@@ -33,11 +33,15 @@ define([
         // in memory from the same user, dismiss the new one
         if (friendRequest[data.msg.author]) { return void cb(true); }
 
-        friendRequest[data.msg.author] = true;
+        friendRequest[data.msg.author] = {
+            type: box.type,
+            hash: data.hash
+        };
 
         // If the user is already in our friend list, automatically accept the request
         if (Messaging.getFriend(ctx.store.proxy, data.msg.author) ||
             ctx.store.proxy.friends_pending[data.msg.author]) {
+
             delete ctx.store.proxy.friends_pending[data.msg.author];
 
             Messaging.acceptFriendRequest(ctx.store, userData, function (obj) {
@@ -49,13 +53,16 @@ define([
                     realtime: ctx.store.realtime,
                     pinPads: ctx.pinPads
                 }, userData, function (err) {
-                    if (err) { return void console.error(err); }
+                    if (err) {
+                        console.error(err);
+                        return void cb(true);
+                    }
                     if (ctx.store.messenger) {
                         ctx.store.messenger.onFriendAdded(userData);
                     }
+                    ctx.updateMetadata();
+                    cb(true);
                 });
-                ctx.updateMetadata();
-                cb(true);
             });
             return;
         }
@@ -79,6 +86,7 @@ define([
     // until the message is manually dismissed.
 
     var friendRequestDeclined = {};
+    var friendRequestAccepted = {};
     handlers['DECLINE_FRIEND_REQUEST'] = function (ctx, box, data, cb) {
         // Old format: data was stored directly in "content"
         var userData = data.msg.content.user || data.msg.content;
@@ -97,27 +105,35 @@ define([
 
             ctx.updateMetadata();
             if (friendRequestDeclined[data.msg.author]) { return; }
-            friendRequestDeclined[data.msg.author] = true;
             box.sendMessage({
                 type: 'FRIEND_REQUEST_DECLINED',
                 content: { user: userData }
-            }, function () {});
+            }, function (hash) {
+                friendRequestDeclined[data.msg.author] = {
+                    type: box.type,
+                    hash: hash
+                };
+            });
         }, getRandomTimeout(ctx));
     };
     // UI for declined friend request
     handlers['FRIEND_REQUEST_DECLINED'] = function (ctx, box, data, cb) {
         ctx.updateMetadata();
         var curve = data.msg.content.user.curvePublic || data.msg.content.user;
-        if (friendRequestDeclined[curve]) { return void cb(true); }
+        var toRemove = friendRequestAccepted[curve];
-        friendRequestDeclined[curve] = true;
+        delete friendRequestAccepted[curve];
-        cb();
+        if (friendRequestDeclined[curve]) { return void cb(true, toRemove); }
+        friendRequestDeclined[curve] = {
+            type: box.type,
+            hash: data.hash
+        };
+        cb(false, toRemove);
     };
     removeHandlers['FRIEND_REQUEST_DECLINED'] = function (ctx, box, data) {
         var curve = data.content.user.curvePublic || data.content.user;
         if (friendRequestDeclined[curve]) { delete friendRequestDeclined[curve]; }
     };
 
-    var friendRequestAccepted = {};
     handlers['ACCEPT_FRIEND_REQUEST'] = function (ctx, box, data, cb) {
         // Old format: data was stored directly in "content"
         var userData = data.msg.content.user || data.msg.content;
@@ -148,11 +164,15 @@ define([
                 if (ctx.store.modules['profile']) { ctx.store.modules['profile'].update(); }
                 // Display the "accepted" state in the UI
                 if (friendRequestAccepted[data.msg.author]) { return; }
-                friendRequestAccepted[data.msg.author] = true;
                 box.sendMessage({
                     type: 'FRIEND_REQUEST_ACCEPTED',
                     content: { user: userData }
-                }, function () {});
+                }, function (hash) {
+                    friendRequestAccepted[data.msg.author] = {
+                        type: box.type,
+                        hash: hash
+                    };
+                });
             });
         }, getRandomTimeout(ctx));
     };
@@ -160,20 +180,32 @@ define([
     handlers['FRIEND_REQUEST_ACCEPTED'] = function (ctx, box, data, cb) {
         ctx.updateMetadata();
         var curve = data.msg.content.user.curvePublic || data.msg.content.user;
-        if (friendRequestAccepted[curve]) { return void cb(true); }
+        var toRemove = friendRequestDeclined[curve];
-        friendRequestAccepted[curve] = true;
+        delete friendRequestDeclined[curve];
-        cb();
+        if (friendRequestAccepted[curve]) { return void cb(true, toRemove); }
+        friendRequestAccepted[curve] = {
+            type: box.type,
+            hash: data.hash
+        };
+        cb(false, toRemove);
     };
     removeHandlers['FRIEND_REQUEST_ACCEPTED'] = function (ctx, box, data) {
         var curve = data.content.user.curvePublic || data.content.user;
         if (friendRequestAccepted[curve]) { delete friendRequestAccepted[curve]; }
     };
 
+    handlers['CANCEL_FRIEND_REQUEST'] = function (ctx, box, data, cb) {
+        var f = friendRequest[data.msg.author];
+        if (!f) { return void cb(true); }
+        cb(true, f);
+    };
+
     handlers['UNFRIEND'] = function (ctx, box, data, cb) {
         var curve = data.msg.author;
         var friend = Messaging.getFriend(ctx.store.proxy, curve);
         if (!friend) { return void cb(true); }
         delete ctx.store.proxy.friends[curve];
+        delete ctx.store.proxy.friends_pending[curve];
         if (ctx.store.messenger) {
             ctx.store.messenger.onFriendRemoved(curve, friend.channel);
         }

www/common/outer/mailbox.js

@@ -271,7 +271,7 @@ proxy.mailboxes = {
                         hash: hash
                     };
                     showMessage(ctx, type, message);
-                    cb();
+                    cb(hash);
                 }, keys.curvePublic);
             };
             box.queue.forEach(function (msg) {
@@ -297,6 +297,7 @@ proxy.mailboxes = {
                     msg: msg,
                     hash: hash
                 };
+                var notify = box.ready;
                 Handlers.add(ctx, box, message, function (dismissed, toDismiss) {
                     if (toDismiss) { // List of other messages to remove
                         dismiss(ctx, toDismiss, '', function () {
@@ -314,8 +315,7 @@ proxy.mailboxes = {
                     }
                     box.content[hash] = msg;
                     showMessage(ctx, type, message, null, function (obj) {
-                        if (!box.ready) { return; }
+                        if (!obj || !obj.msg || !notify) { return; }
-                        if (!obj || !obj.msg) { return; }
                         Notify.system(undefined, obj.msg);
                     });
                 });

www/common/outer/messenger.js

@@ -459,6 +459,13 @@ define([
         }
     };
 
+    // Cancel pending friend requests
+    var cancelFriend = function (ctx, data, _cb) {
+        var cb = Util.once(_cb);
+        if (typeof(cb) !== 'function') { return void console.error('NO_CALLBACK'); }
+        ctx.Store.cancelFriendRequest(data, cb);
+    };
+
     var getAllClients = function (ctx) {
         var all = [];
         Array.prototype.push.apply(all, ctx.friendsClients);
@@ -935,6 +942,7 @@ define([
         if (AppConfig.availablePadTypes.indexOf('contacts') === -1) { return; }
         var ctx = {
             store: store,
+            Store: cfg.Store,
             updateMetadata: cfg.updateMetadata,
             pinPads: cfg.pinPads,
             emit: emit,
@@ -1047,6 +1055,9 @@ define([
             if (cmd === 'REMOVE_FRIEND') {
                 return void removeFriend(ctx, data, cb);
             }
+            if (cmd === 'CANCEL_FRIEND') {
+                return void cancelFriend(ctx, data, cb);
+            }
             if (cmd === 'MUTE_USER') {
                 return void muteUser(ctx, data, cb);
             }

www/common/outer/roster.js

@@ -1,5 +1,5 @@
 (function () {
-var factory = function (Util, Hash, CPNetflux, Sortify, nThen, Crypto) {
+var factory = function (Util, Hash, CPNetflux, Sortify, nThen, Crypto, Feedback) {
     var Roster = {};
 
     // this constant is somewhat arbitrary.
@@ -587,6 +587,11 @@ var factory = function (Util, Hash, CPNetflux, Sortify, nThen, Crypto) {
         // deleted while you are open
         // emit an event
         var onChannelError = function (info) {
+            if (Feedback) { Feedback.send('ROSTER_CHANNEL_ERROR='+(info && info.type)); }
+            if (info && info.type === "EUNKNOWN") {
+                // chainpad-netflux should recover by itself
+                return;
+            }
             if (!ready) { return void cb(info); }
             console.error("CHANNEL_ERROR", info);
         };
@@ -870,7 +875,8 @@ var factory = function (Util, Hash, CPNetflux, Sortify, nThen, Crypto) {
             require("../../bower_components/chainpad-netflux/chainpad-netflux.js"),
             require("../../bower_components/json.sortify"),
             require("nthen"),
-            require("../../bower_components/chainpad-crypto/crypto")
+            require("../../bower_components/chainpad-crypto/crypto"),
+            null // no feedback here
         );
     } else if ((typeof(define) !== 'undefined' && define !== null) && (define.amd !== null)) {
         require.config({ paths:  { 'json.sortify': '/bower_components/json.sortify/dist/JSON.sortify' } });
@@ -880,16 +886,18 @@ var factory = function (Util, Hash, CPNetflux, Sortify, nThen, Crypto) {
             '/bower_components/chainpad-netflux/chainpad-netflux.js',
             'json.sortify',
             '/bower_components/nthen/index.js',
-            '/bower_components/chainpad-crypto/crypto.js'
+            '/bower_components/chainpad-crypto/crypto.js',
+            '/common/common-feedback.js',
             //'/bower_components/tweetnacl/nacl-fast.min.js',
-        ], function (Util, Hash, CPNF, Sortify, nThen, Crypto) {
+        ], function (Util, Hash, CPNF, Sortify, nThen, Crypto, Feedback) {
             return factory.apply(null, [
                 Util,
                 Hash,
                 CPNF,
                 Sortify,
                 nThen,
-                Crypto
+                Crypto,
+                Feedback
             ]);
         });
     } else {

www/common/outer/sharedfolder.js

@@ -2,12 +2,13 @@ define([
     '/common/common-hash.js',
     '/common/common-util.js',
     '/common/userObject.js',
+    '/common/outer/cache-store.js',
 
     '/bower_components/nthen/index.js',
     '/bower_components/chainpad-crypto/crypto.js',
     '/bower_components/chainpad-listmap/chainpad-listmap.js',
     '/bower_components/chainpad/chainpad.dist.js',
-], function (Hash, Util, UserObject,
+], function (Hash, Util, UserObject, Cache,
              nThen, Crypto, Listmap, ChainPad) {
     var SF = {};
 
@@ -174,6 +175,7 @@ define([
                 ChainPad: ChainPad,
                 classic: true,
                 network: network,
+                Cache: Cache, // XXX re-enable cache usage
                 metadata: {
                     validateKey: secret.keys.validateKey || undefined,
                     owners: owners

www/common/outer/store-rpc.js

@@ -88,6 +88,7 @@ define([
             CHANGE_PAD_PASSWORD_PIN: Store.changePadPasswordPin,
             GET_LAST_HASH: Store.getLastHash,
             GET_SNAPSHOT: Store.getSnapshot,
+            CORRUPTED_CACHE: Store.corruptedCache,
             // Drive
             DRIVE_USEROBJECT: Store.userObjectCommand,
             // Settings,

www/common/outer/team.js

@@ -12,6 +12,7 @@ define([
     '/common/common-feedback.js',
     '/common/outer/invitation.js',
     '/common/cryptget.js',
+    '/common/outer/cache-store.js',
 
     '/bower_components/chainpad-listmap/chainpad-listmap.js',
     '/bower_components/chainpad-crypto/crypto.js',
@@ -21,7 +22,7 @@ define([
     '/bower_components/saferphore/index.js',
     '/bower_components/tweetnacl/nacl-fast.min.js',
 ], function (Util, Hash, Constants, Realtime,
-             ProxyManager, UserObject, SF, Roster, Messaging, Feedback, Invite, Crypt,
+             ProxyManager, UserObject, SF, Roster, Messaging, Feedback, Invite, Crypt, Cache,
              Listmap, Crypto, CpNetflux, ChainPad, nThen, Saferphore) {
     var Team = {};
 
@@ -57,11 +58,11 @@ define([
             });
             proxy.on('disconnect', function () {
                 team.offline = true;
-                team.sendEvent('NETWORK_DISCONNECT');
+                team.sendEvent('NETWORK_DISCONNECT', team.id);
             });
             proxy.on('reconnect', function () {
                 team.offline = false;
-                team.sendEvent('NETWORK_RECONNECT');
+                team.sendEvent('NETWORK_RECONNECT', team.id);
             });
         }
         proxy.on('change', [], function (o, n, p) {
@@ -426,6 +427,7 @@ define([
                 channel: secret.channel,
                 crypto: crypto,
                 ChainPad: ChainPad,
+                Cache: Cache, // XXX re-enable cache usage
                 metadata: {
                     validateKey: secret.keys.validateKey || undefined,
                 },
@@ -573,6 +575,7 @@ define([
             logLevel: 1,
             classic: true,
             ChainPad: ChainPad,
+            Cache: Cache,
             owners: [ctx.store.proxy.edPublic]
         };
         nThen(function (waitFor) {
@@ -931,7 +934,9 @@ define([
         if (!team) { return void cb ({error: 'ENOENT'}); }
         if (!team.roster) { return void cb({error: 'NO_ROSTER'}); }
         var state = team.roster.getState() || {};
-        cb(state.metadata || {});
+        var md = state.metadata || {};
+        md.offline = team.offline;
+        cb(md);
     };
 
     var setTeamMetadata = function (ctx, data, cId, cb) {
@@ -1791,7 +1796,11 @@ define([
                 teams[id].keys.mailbox = deriveMailbox(teams[id]);
             }
             openChannel(ctx, teams[id], id, waitFor(function (err) {
-                if (err) { return void console.error(err); }
+                if (err) {
+                    var txt = typeof(err) === "string" ? err : (err.type || err.message);
+                    Feedback.send("TEAM_LOADING_ERROR="+txt);
+                    return void console.error(err);
+                }
                 console.debug('Team '+id+' ready');
             }));
         });
@@ -1879,15 +1888,15 @@ define([
             var t = Util.clone(teams);
             Object.keys(t).forEach(function (id) {
                 // If failure to load the team, don't send it
-                if (ctx.teams[id]) { return; }
+                if (ctx.teams[id]) {
+                    t[id].offline = ctx.teams[id].offline;
+                    return;
+                }
                 t[id].error = true;
             });
             cb(t);
         };
         team.execCommand = function (clientId, obj, cb) {
-            if (ctx.store.offline) {
-                return void cb({ error: 'OFFLINE' });
-            }
 
             var cmd = obj.cmd;
             var data = obj.data;
@@ -1911,30 +1920,36 @@ define([
                 return void setTeamMetadata(ctx, data, clientId, cb);
             }
             if (cmd === 'OFFER_OWNERSHIP') {
+                if (ctx.store.offline) { return void cb({ error: 'OFFLINE' }); }
                 return void offerOwnership(ctx, data, clientId, cb);
             }
             if (cmd === 'ANSWER_OWNERSHIP') {
+                if (ctx.store.offline) { return void cb({ error: 'OFFLINE' }); }
                 return void answerOwnership(ctx, data, clientId, cb);
             }
             if (cmd === 'DESCRIBE_USER') {
                 return void describeUser(ctx, data, clientId, cb);
             }
             if (cmd === 'INVITE_TO_TEAM') {
+                if (ctx.store.offline) { return void cb({ error: 'OFFLINE' }); }
                 return void inviteToTeam(ctx, data, clientId, cb);
             }
             if (cmd === 'LEAVE_TEAM') {
                 return void leaveTeam(ctx, data, clientId, cb);
             }
             if (cmd === 'JOIN_TEAM') {
+                if (ctx.store.offline) { return void cb({ error: 'OFFLINE' }); }
                 return void joinTeam(ctx, data, clientId, cb);
             }
             if (cmd === 'REMOVE_USER') {
                 return void removeUser(ctx, data, clientId, cb);
             }
             if (cmd === 'DELETE_TEAM') {
+                if (ctx.store.offline) { return void cb({ error: 'OFFLINE' }); }
                 return void deleteTeam(ctx, data, clientId, cb);
             }
             if (cmd === 'CREATE_TEAM') {
+                if (ctx.store.offline) { return void cb({ error: 'OFFLINE' }); }
                 return void createTeam(ctx, data, clientId, cb);
             }
             if (cmd === 'GET_EDITABLE_FOLDERS') {
@@ -1947,6 +1962,7 @@ define([
                 return void getPreviewContent(ctx, data, clientId, cb);
             }
             if (cmd === 'ACCEPT_LINK_INVITATION') {
+                if (ctx.store.offline) { return void cb({ error: 'OFFLINE' }); }
                 return void acceptLinkInvitation(ctx, data, clientId, cb);
             }
         };

www/common/outer/upload.js

@@ -1,9 +1,11 @@
 define([
     '/file/file-crypto.js',
     '/common/common-hash.js',
+    '/common/common-util.js',
+    '/common/outer/cache-store.js',
     '/bower_components/nthen/index.js',
     '/bower_components/tweetnacl/nacl-fast.min.js',
-], function (FileCrypto, Hash, nThen) {
+], function (FileCrypto, Hash, Util, Cache, nThen) {
     var Nacl = window.nacl;
     var module = {};
 
@@ -31,9 +33,11 @@ define([
         };
 
         var actual = 0;
+        var encryptedArr = [];
         var again = function (err, box) {
             if (err) { onError(err); }
             if (box) {
+                encryptedArr.push(box);
                 actual += box.length;
                 var progressValue = (actual / estimate * 100);
                 progressValue = Math.min(progressValue, 100);
@@ -55,9 +59,11 @@ define([
                 var uri = ['', 'blob', id.slice(0,2), id].join('/');
                 console.log("encrypted blob is now available as %s", uri);
 
-
+                var box_u8 = Util.uint8ArrayJoin(encryptedArr);
-
+                Cache.setBlobCache(id, box_u8, function (err) {
-                cb();
+                    if (err) { console.warn(err); }
+                    cb();
+                });
             });
         };
 

www/common/outer/worker-channel.js

@@ -65,11 +65,13 @@ define([
                 cb(undefined, data.content, msg);
             };
             evReady.reg(function () {
-                postMsg(JSON.stringify({
+                var toSend = {
                     txid: txid,
                     content: content,
-                    q: q
+                    q: q,
-                }));
+                    raw: opts.raw
+                };
+                postMsg(opts.raw ? toSend : JSON.stringify(toSend));
             });
         };
 
@@ -84,12 +86,13 @@ define([
         // If the type is a query, your handler will be invoked with a reply function that takes
         // one argument (the content to reply with).
         chan.on = function (queryType, handler, quiet) {
-            var h = function (data, msg) {
+            var h = function (data, msg, raw) {
                 handler(data.content, function (replyContent) {
-                    postMsg(JSON.stringify({
+                    var toSend = {
                         txid: data.txid,
                         content: replyContent
-                    }));
+                    };
+                    postMsg(raw ? toSend : JSON.stringify(toSend));
                 }, msg);
             };
             (handlers[queryType] = handlers[queryType] || []).push(h);
@@ -150,7 +153,7 @@ define([
         onMsg.reg(function (msg) {
             if (!chanLoaded) { return; }
             if (!msg.data || msg.data === '_READY') { return; }
-            var data = JSON.parse(msg.data);
+            var data = typeof(msg.data) === "object" ? msg.data : JSON.parse(msg.data);
             if (typeof(data.ack) !== "undefined") {
                 if (acks[data.txid]) { acks[data.txid](!data.ack); }
             } else if (typeof(data.q) === 'string') {
@@ -163,7 +166,7 @@ define([
                         }));
                     }
                     handlers[data.q].forEach(function (f) {
-                        f(data || JSON.parse(msg.data), msg);
+                        f(data || JSON.parse(msg.data), msg, data && data.raw);
                         data = undefined;
                     });
                 } else {

www/common/proxy-manager.js

@@ -40,6 +40,14 @@ define([
             userObject: userObject,
             leave: leave
         };
+        if (proxy.on) {
+            proxy.on('disconnect', function () {
+                Env.folders[id].offline = true;
+            });
+            proxy.on('reconnect', function () {
+                Env.folders[id].offline = false;
+            });
+        }
         return userObject;
     };
 

www/common/sframe-app-framework.js

@@ -221,6 +221,10 @@ define([
                     evStart.reg(function () { toolbar.deleted(); });
                     break;
                 }
+                case STATE.READY: {
+                    evStart.reg(function () { toolbar.ready(); });
+                    break;
+                }
                 default:
             }
             var isEditable = (state === STATE.READY && !unsyncMode);
@@ -467,7 +471,51 @@ define([
             });
         };
 
+        var noCache = false; // Prevent reload loops
+        var onCorruptedCache = function () {
+            if (noCache) {
+                UI.errorLoadingScreen(Messages.unableToDisplay, false, function () {
+                    common.gotoURL('');
+                });
+            }
+            noCache = true;
+            var sframeChan = common.getSframeChannel();
+            sframeChan.event("EV_CORRUPTED_CACHE");
+        };
+        var onCacheReady = function () {
+            stateChange(STATE.INITIALIZING);
+            toolbar.offline(true);
+            var newContentStr = cpNfInner.chainpad.getUserDoc();
+            if (toolbar) {
+                // Check if we have a new chainpad instance
+                toolbar.resetChainpad(cpNfInner.chainpad);
+            }
+
+            // Invalid cache
+            if (newContentStr === '') { return void onCorruptedCache(); }
+
+            var privateDat = cpNfInner.metadataMgr.getPrivateData();
+            var type = privateDat.app;
+
+            var newContent = JSON.parse(newContentStr);
+            var metadata = extractMetadata(newContent);
+
+            // Make sure we're using the correct app for this cache
+            if (metadata && typeof(metadata.type) !== 'undefined' && metadata.type !== type) {
+                return void onCorruptedCache();
+            }
+
+            cpNfInner.metadataMgr.updateMetadata(metadata);
+            newContent = normalize(newContent);
+            if (!unsyncMode) {
+                contentUpdate(newContent, function () { return function () {}; });
+            }
+
+            UI.removeLoadingScreen(emitResize);
+        };
         var onReady = function () {
+            toolbar.offline(false);
+
             var newContentStr = cpNfInner.chainpad.getUserDoc();
             if (state === STATE.DELETED) { return; }
 
@@ -482,7 +530,6 @@ define([
             var privateDat = cpNfInner.metadataMgr.getPrivateData();
             var type = privateDat.app;
 
-
             // contentUpdate may be async so we need an nthen here
             nThen(function (waitFor) {
                 if (!newPad) {
@@ -508,14 +555,19 @@ define([
                         console.log("Either this is an empty document which has not been touched");
                         console.log("Or else something is terribly wrong, reloading.");
                         Feedback.send("NON_EMPTY_NEWDOC");
-                        setTimeout(function () { common.gotoURL(); }, 1000);
+                        // The cache may be wrong, empty it and reload after.
+                        waitFor.abort();
+                        onCorruptedCache();
                         return;
                     }
-                    console.log('updating title');
                     title.updateTitle(title.defaultTitle);
                     evOnDefaultContentNeeded.fire();
                 }
             }).nThen(function () {
+                // We have a valid chainpad, reenable cache fix in case with reconnect with
+                // a corrupted cache
+                noCache = false;
+
                 stateChange(STATE.READY);
                 firstConnection = false;
 
@@ -551,6 +603,8 @@ define([
                         Thumb.initPadThumbnails(common, options.thumbnail);
                     }
                 }
+
+                common.checkTrimHistory();
             });
         };
         var onConnectionChange = function (info) {
@@ -732,6 +786,7 @@ define([
                 onRemote: onRemote,
                 onLocal: onLocal,
                 onInit: onInit,
+                onCacheReady: function () { evStart.reg(onCacheReady); },
                 onReady: function () { evStart.reg(onReady); },
                 onConnectionChange: onConnectionChange,
                 onError: onError,

www/common/sframe-chainpad-netflux-inner.js

@@ -34,6 +34,7 @@ define([
         var onLocal = config.onLocal || function () { };
         var setMyID = config.setMyID || function () { };
         var onReady = config.onReady || function () { };
+        var onCacheReady = config.onCacheReady || function () { };
         var onError = config.onError || function () { };
         var userName = config.userName;
         var initialState = config.initialState;
@@ -93,6 +94,9 @@ define([
             evInfiniteSpinner.fire();
         }, 2000);
 
+        sframeChan.on('EV_RT_CACHE_READY', function () {
+            onCacheReady({realtime: chainpad});
+        });
         sframeChan.on('EV_RT_DISCONNECT', function (isPermanent) {
             isReady = false;
             chainpad.abort();

www/common/sframe-chainpad-netflux-outer.js

@@ -46,6 +46,7 @@ define([], function () {
         // shim between chainpad and netflux
         var msgIn = function (peer, msg) {
             try {
+                if (/^\[/.test(msg)) { return msg; } // Already decrypted
                 var isHk = peer.length !== 32;
                 var key = isNewHash ? validateKey : false;
                 var decryptedMsg = Crypto.decrypt(msg, key, isHk);
@@ -114,16 +115,25 @@ define([], function () {
             if (firstConnection) {
                 firstConnection = false;
                 // Add the handlers to the WebChannel
-                padRpc.onMessageEvent.reg(function (msg) { onMessage(msg); });
                 padRpc.onJoinEvent.reg(function (m) { sframeChan.event('EV_RT_JOIN', m); });
                 padRpc.onLeaveEvent.reg(function (m) { sframeChan.event('EV_RT_LEAVE', m); });
             }
         };
 
+        padRpc.onMessageEvent.reg(function (msg) { onMessage(msg); });
+
         padRpc.onDisconnectEvent.reg(function (permanent) {
             sframeChan.event('EV_RT_DISCONNECT', permanent);
         });
 
+        padRpc.onCacheReadyEvent.reg(function () {
+            sframeChan.event('EV_RT_CACHE_READY');
+        });
+
+        padRpc.onCacheEvent.reg(function () {
+            sframeChan.event('EV_RT_CACHE');
+        });
+
         padRpc.onConnectEvent.reg(function (data) {
             onOpen(data);
         });

www/common/sframe-common-file.js

@@ -1,5 +1,6 @@
 define([
     'jquery',
+    '/api/config',
     '/file/file-crypto.js',
     '/common/make-backup.js',
     '/common/common-thumbnail.js',
@@ -12,7 +13,7 @@ define([
 
     '/bower_components/file-saver/FileSaver.min.js',
     '/bower_components/tweetnacl/nacl-fast.min.js',
-], function ($, FileCrypto, MakeBackup, Thumb, UI, UIElements, Util, Hash, h, Messages) {
+], function ($, ApiConfig, FileCrypto, MakeBackup, Thumb, UI, UIElements, Util, Hash, h, Messages) {
     var Nacl = window.nacl;
     var module = {};
 
@@ -48,7 +49,7 @@ define([
         };
 
         var tableHeader = h('div.cp-fileupload-header', [
-            h('div.cp-fileupload-header-title', h('span', Messages.fileuploadHeader || 'Uploaded files')),
+            h('div.cp-fileupload-header-title', h('span', Messages.fileTableHeader)),
             h('div.cp-fileupload-header-close', h('span.fa.fa-times')),
         ]);
 
@@ -136,13 +137,11 @@ define([
 
             file.uid = Util.uid();
             response.expect(file.uid, function (href) {
-                var mdMgr = common.getMetadataMgr();
-                var origin = mdMgr.getPrivateData().origin;
                 $link.prepend($('<span>', {'class': 'fa fa-external-link'}));
                 $link.attr('href', href)
                     .click(function (e) {
                         e.preventDefault();
-                        window.open(origin + $link.attr('href'), '_blank');
+                        common.openURL($link.attr('href'));
                     });
                 var title = metadata.name;
                 if (!config.noStore) {
@@ -168,8 +167,12 @@ define([
                 if (config.onError) { config.onError(e); }
 
                 if (e === 'TOO_LARGE') {
-                    $pv.text(Messages.upload_tooLargeBrief);
+                    var privateData = common.getMetadataMgr().getPrivateData();
-                    return void UI.alert(Messages.upload_tooLarge);
+                    var l = privateData.plan ? ApiConfig.premiumUploadSize : false;
+                    l = l || ApiConfig.maxUploadSize || '?';
+                    var maxSizeStr = Util.bytesToMegabytes(l);
+                    $pv.text(Messages.error);
+                    return void UI.alert(Messages._getKey('upload_tooLargeBrief', [maxSizeStr]));
                 }
                 if (e === 'NOT_ENOUGH_SPACE') {
                     $pv.text(Messages.upload_notEnoughSpaceBrief);
@@ -262,7 +265,8 @@ define([
             // name
             $('<td>').append($link).appendTo($tr);
             // size
-            $('<td>').text(UIElements.prettySize(estimate)).appendTo($tr);
+            var size = estimate ? UIElements.prettySize(estimate) : '';
+            $(h('td.cp-fileupload-size')).text(size).appendTo($tr);
             // progress
             $('<td>', {'class': 'cp-fileupload-table-progress'}).append($progressContainer).appendTo($tr);
             // cancel
@@ -590,12 +594,11 @@ define([
                 queue.next();
             };
 
-            /*
             var cancelled = function () {
                 $row.find('.cp-fileupload-table-cancel').addClass('cancelled').html('').append(h('span.fa.fa-minus'));
                 queue.inProgress = false;
                 queue.next();
-            };*/
+            };
 
             /**
              * Update progress in the download panel, for downloading a file
@@ -629,6 +632,17 @@ define([
              */
             var updateProgress = function (progressValue) {
                 var text = Math.round(progressValue*100) + '%';
+                if (Array.isArray(data.list)) {
+                    text = Messages._getKey('download_zip_file', [Math.round(progressValue * data.list.length), data.list.length]);
+                }
+                if (progressValue === 2) {
+                    text = Messages.download_zip;
+                    progressValue = 1;
+                }
+                if (progressValue === 3) {
+                    text = "100%";
+                    progressValue = 1;
+                }
                 $pv.text(text);
                 $pb.css({
                     width: (progressValue * 100) + '%'
@@ -640,8 +654,10 @@ define([
                 fileHost: privateData.fileHost,
                 get: common.getPad,
                 sframeChan: sframeChan,
+                cache: common.getCache()
             };
-            downloadFunction(ctx, data, function (err, obj) {
+
+            var dl = downloadFunction(ctx, data, function (err, obj) {
                 $link.prepend($('<span>', {'class': 'fa fa-external-link'}))
                     .attr('href', '#')
                     .click(function (e) {
@@ -657,19 +673,17 @@ define([
                 folderProgress: updateProgress,
             });
 
-/*
+            var $cancel = $row.find('.cp-fileupload-table-cancel').html('');
-            var $cancel = $('<span>', {'class': 'cp-fileupload-table-cancel-button fa fa-times'}).click(function () {
+            if (dl && dl.cancel) {
-                dl.cancel();
+                $('<span>', {
-                $cancel.remove();
+                    'class': 'cp-fileupload-table-cancel-button fa fa-times'
-                $row.find('.cp-fileupload-table-progress-value').text(Messages.upload_cancelled);
+                }).click(function () {
-                cancelled();
+                    dl.cancel();
-            });
+                    $cancel.remove();
-*/
+                    $row.find('.cp-fileupload-table-progress-value').text(Messages.upload_cancelled);
-
+                    cancelled();
-            $row.find('.cp-fileupload-table-cancel')
+                }).appendTo($cancel);
-                .html('')
+            }
-                .append(h('span.fa.fa-minus'));
-                //.append($cancel);
         };
 
         File.downloadFile = function (fData, cb) {

www/common/sframe-common-outer.js

@@ -8,7 +8,7 @@ define([
 ], function (nThen, ApiConfig, RequireConfig, Messages, $) {
     var common = {};
 
-    common.initIframe = function (waitFor, isRt) {
+    common.initIframe = function (waitFor, isRt, pathname) {
         var requireConfig = RequireConfig();
         var lang = Messages._languageUsed;
         var req = {
@@ -31,7 +31,7 @@ define([
         }
 
         document.getElementById('sbox-iframe').setAttribute('src',
-            ApiConfig.httpSafeOrigin + window.location.pathname + 'inner.html?' +
+            ApiConfig.httpSafeOrigin + (pathname || window.location.pathname) + 'inner.html?' +
                 requireConfig.urlArgs + '#' + encodeURIComponent(JSON.stringify(req)));
 
         // This is a cheap trick to avoid loading sframe-channel in parallel with the
@@ -97,15 +97,17 @@ define([
                 '/common/common-hash.js',
                 '/common/common-util.js',
                 '/common/common-realtime.js',
+                '/common/notify.js',
                 '/common/common-constants.js',
                 '/common/common-feedback.js',
                 '/common/outer/local-store.js',
+                '/common/outer/cache-store.js',
                 '/customize/application_config.js',
                 '/common/test.js',
                 '/common/userObject.js',
             ], waitFor(function (_CpNfOuter, _Cryptpad, _Crypto, _Cryptget, _SFrameChannel,
-            _SecureIframe, _Messaging, _Notifier, _Hash, _Util, _Realtime,
+            _SecureIframe, _Messaging, _Notifier, _Hash, _Util, _Realtime, _Notify,
-            _Constants, _Feedback, _LocalStore, _AppConfig, _Test, _UserObject) {
+            _Constants, _Feedback, _LocalStore, _Cache, _AppConfig, _Test, _UserObject) {
                 CpNfOuter = _CpNfOuter;
                 Cryptpad = _Cryptpad;
                 Crypto = Utils.Crypto = _Crypto;
@@ -120,7 +122,9 @@ define([
                 Utils.Constants = _Constants;
                 Utils.Feedback = _Feedback;
                 Utils.LocalStore = _LocalStore;
+                Utils.Cache = _Cache;
                 Utils.UserObject = _UserObject;
+                Utils.Notify = _Notify;
                 Utils.currentPad = currentPad;
                 AppConfig = _AppConfig;
                 Test = _Test;
@@ -478,6 +482,7 @@ define([
                             // We've received a link without /p/ and it doesn't work without a password: abort
                             return void todo();
                         }
+
                         // Wrong password or deleted file?
                         askPassword(true, passwordCfg);
                     }));
@@ -532,6 +537,12 @@ define([
             var edPublic, curvePublic, notifications, isTemplate;
             var settings = {};
             var isSafe = ['debug', 'profile', 'drive', 'teams'].indexOf(currentPad.app) !== -1;
+
+            var isDeleted = isNewFile && currentPad.hash.length > 0;
+            if (isDeleted) {
+                Utils.Cache.clearChannel(secret.channel);
+            }
+
             var updateMeta = function () {
                 //console.log('EV_METADATA_UPDATE');
                 var metaObj;
@@ -555,6 +566,7 @@ define([
                         defaultTitle: defaultTitle,
                         type: cfg.type || parsed.type
                     };
+                    var notifs = Utils.Notify.isSupported() && Utils.Notify.hasPermission();
                     var additionalPriv = {
                         app: parsed.type,
                         loggedIn: Utils.LocalStore.isLoggedIn(),
@@ -569,12 +581,13 @@ define([
                         isPresent: parsed.hashData && parsed.hashData.present,
                         isEmbed: parsed.hashData && parsed.hashData.embed,
                         isHistoryVersion: parsed.hashData && parsed.hashData.versionHash,
+                        notifications: notifs,
                         accounts: {
                             donateURL: Cryptpad.donateURL,
                             upgradeURL: Cryptpad.upgradeURL
                         },
                         isNewFile: isNewFile,
-                        isDeleted: isNewFile && currentPad.hash.length > 0,
+                        isDeleted: isDeleted,
                         password: password,
                         channel: secret.channel,
                         enableSF: localStorage.CryptPad_SF === "1", // TODO to remove when enabled by default
@@ -677,6 +690,22 @@ define([
                     });
                 });
 
+                sframeChan.on('Q_GET_BLOB_CACHE', function (data, cb) {
+                    if (!Utils.Cache) { return void cb({error: 'NOCACHE'}); }
+                    Utils.Cache.getBlobCache(data.id, function (err, obj) {
+                        if (err) { return void cb({error: err}); }
+                        cb(obj);
+                    });
+                });
+                sframeChan.on('Q_SET_BLOB_CACHE', function (data, cb) {
+                    if (!Utils.Cache) { return void cb({error: 'NOCACHE'}); }
+                    if (!data || !data.u8 || typeof(data.u8) !== "object") { return void cb({error: 'EINVAL'}); }
+                    Utils.Cache.setBlobCache(data.id, data.u8, function (err) {
+                        if (err) { return void cb({error: err}); }
+                        cb();
+                    });
+                });
+
                 sframeChan.on('Q_GET_ATTRIBUTE', function (data, cb) {
                     Cryptpad.getAttribute(data.key, function (e, data) {
                         cb({
@@ -722,7 +751,16 @@ define([
 
                 sframeChan.on('EV_OPEN_URL', function (url) {
                     if (url) {
-                        window.open(url);
+                        var a = window.open(url);
+                        if (!a) {
+                            sframeChan.event('EV_POPUP_BLOCKED');
+                        }
+                    }
+                });
+
+                sframeChan.on('EV_OPEN_UNSAFE_URL', function (url) {
+                    if (url) {
+                        window.open(ApiConfig.httpSafeOrigin + '/bounce/#' + encodeURIComponent(url));
                     }
                 });
 
@@ -1569,9 +1607,14 @@ define([
                 });
             });
 
-            if (cfg.messaging) {
+            sframeChan.on('Q_ASK_NOTIFICATION', function (data, cb) {
-                Notifier.getPermission();
+                if (!Utils.Notify.isSupported()) { return void cb(false); }
+                Notification.requestPermission(function (s) {
+                    cb(s === "granted");
+                });
+            });
 
+            if (cfg.messaging) {
                 sframeChan.on('Q_CHAT_OPENPADCHAT', function (data, cb) {
                     Cryptpad.universal.execCommand({
                         type: 'messenger',
@@ -1683,6 +1726,10 @@ define([
                 });
             };
 
+            sframeChan.on('EV_CORRUPTED_CACHE', function () {
+                Cryptpad.onCorruptedCache(secret.channel);
+            });
+
             sframeChan.on('Q_CREATE_PAD', function (data, cb) {
                 if (!isNewFile || rtStarted) { return; }
                 // Create a new hash
@@ -1797,7 +1844,12 @@ define([
                             }
                             startRealtime();
                             cb();
-                        }, cryptputCfg);
+                        }, cryptputCfg, function (progress) {
+                            sframeChan.event('EV_LOADING_INFO', {
+                                type: 'pad',
+                                progress: progress
+                            });
+                        });
                         return;
                     }
                     // Start realtime outside the iframe and callback

www/common/sframe-common.js

@@ -11,6 +11,7 @@ define([
     '/common/sframe-common-codemirror.js',
     '/common/sframe-common-cursor.js',
     '/common/sframe-common-mailbox.js',
+    '/common/inner/cache.js',
     '/common/inner/common-mediatag.js',
     '/common/metadata-manager.js',
 
@@ -36,6 +37,7 @@ define([
     CodeMirror,
     Cursor,
     Mailbox,
+    Cache,
     MT,
     MetadataMgr,
     AppConfig,
@@ -142,7 +144,7 @@ define([
         }
         return;
     };
-    funcs.importMediaTag = function ($mt) {
+    var getMtData = function ($mt) {
         if (!$mt || !$mt.is('media-tag')) { return; }
         var chanStr = $mt.attr('src');
         var keyStr = $mt.attr('data-crypto-key');
@@ -154,10 +156,27 @@ define([
         var channel = src.replace(/\/blob\/[0-9a-f]{2}\//i, '');
         // Get key
         var key = keyStr.replace(/cryptpad:/i, '');
+        return {
+            channel: channel,
+            key: key
+        };
+    };
+    funcs.getHashFromMediaTag = function ($mt) {
+        var data = getMtData($mt);
+        if (!data) { return; }
+        return Hash.getFileHashFromKeys({
+            version: 1,
+            channel: data.channel,
+            keys: { fileKeyStr: data.key }
+        });
+    };
+    funcs.importMediaTag = function ($mt) {
+        var data = getMtData($mt);
+        if (!data) { return; }
         var metadata = $mt[0]._mediaObject._blob.metadata;
         ctx.sframeChan.query('Q_IMPORT_MEDIATAG', {
-            channel: channel,
+            channel: data.channel,
-            key: key,
+            key: data.key,
             name: metadata.name,
             type: metadata.type,
             owners: metadata.owners
@@ -264,6 +283,65 @@ define([
         return teamChatChannel;
     };
 
+    // When opening a pad, if were an owner check the history size and prompt for trimming if
+    // necessary
+    funcs.checkTrimHistory = function (channels, isDrive) {
+        channels = channels || [];
+        var priv = ctx.metadataMgr.getPrivateData();
+
+        var limit = 100 * 1024 * 1024; // 100MB
+
+        var owned;
+        nThen(function (w) {
+            if (isDrive) {
+                funcs.getAttribute(['drive', 'trim'], w(function (err, val) {
+                    if (err || typeof(val) !== "number") { return; }
+                    if (val < (+new Date())) { return; }
+                    w.abort();
+                }));
+                return;
+            }
+            funcs.getPadAttribute('trim', w(function (err, val) {
+                if (err || typeof(val) !== "number") { return; }
+                if (val < (+new Date())) { return; }
+                w.abort();
+            }));
+        }).nThen(function (w) {
+            // Check ownership
+            // DRIVE
+            if (isDrive) {
+                if (!priv.isDriveOwned) { return void w.abort(); }
+                return;
+            }
+            // PAD
+            channels.push({ channel: priv.channel });
+            funcs.getPadMetadata({
+                channel: priv.channel
+            }, w(function (md) {
+                if (md && md.error) { return void w.abort(); }
+                var owners = md.owners;
+                owned = funcs.isOwned(owners);
+                if (!owned) { return void w.abort(); }
+            }));
+        }).nThen(function () {
+            // We're an owner: check the history size
+            var history = funcs.makeUniversal('history');
+            history.execCommand('GET_HISTORY_SIZE', {
+                account: isDrive,
+                pad: !isDrive,
+                channels: channels,
+                teamId: typeof(owned) === "number" && owned
+            }, function (obj) {
+                if (obj && obj.error) { return; } // can't get history size: abort
+                var bytes = obj.size;
+                if (!bytes || typeof(bytes) !== "number") { return; } // no history: abort
+                if (bytes < limit) { return; }
+                obj.drive = isDrive;
+                UIElements.displayTrimHistoryPrompt(funcs, obj);
+            });
+        });
+    };
+
     var cursorChannel;
     // common-ui-elements needs to be able to get the cursor channel to put it in metadata when
     // importing a template
@@ -528,6 +606,10 @@ define([
         });
     };
 
+    funcs.getCache = function () {
+        return ctx.cache;
+    };
+
 /*    funcs.storeLinkToClipboard = function (readOnly, cb) {
         ctx.sframeChan.query('Q_STORE_LINK_TO_CLIPBOARD', readOnly, function (err) {
             if (cb) { cb(err); }
@@ -555,7 +637,11 @@ define([
         return window.location.origin + '/bounce/#' + encodeURIComponent(url);
     };
     funcs.openUnsafeURL = function (url) {
-        var bounceHref = funcs.getBounceURL(url);
+        var app = ctx.metadataMgr.getPrivateData().app;
+        if (app === "sheet") {
+            return void ctx.sframeChan.event('EV_OPEN_UNSAFE_URL', url);
+        }
+        var bounceHref = window.location.origin + '/bounce/#' + encodeURIComponent(url);
         window.open(bounceHref);
     };
 
@@ -685,6 +771,10 @@ define([
                 UI.errorLoadingScreen(Messages.password_error_seed);
             });
 
+            ctx.sframeChan.on("EV_POPUP_BLOCKED", function () {
+                UI.alert(Messages.errorPopupBlocked);
+            });
+
             ctx.sframeChan.on("EV_EXPIRED_ERROR", function () {
                 funcs.onServerError({
                     type: 'EEXPIRED'
@@ -729,12 +819,24 @@ define([
                 modules[type].onEvent(obj.data);
             });
 
+            ctx.cache = Cache.create(ctx.sframeChan);
+
             ctx.metadataMgr.onReady(waitFor());
 
         }).nThen(function () {
             var privateData = ctx.metadataMgr.getPrivateData();
             funcs.addShortcuts(window, Boolean(privateData.app));
 
+            var mt = Util.find(privateData, ['settings', 'general', 'mediatag-size']);
+            if (MT.MediaTag && typeof(mt) === "number") {
+                var maxMtSize = mt === -1 ? Infinity : mt * 1024 * 1024;
+                MT.MediaTag.setDefaultConfig('maxDownloadSize', maxMtSize);
+            }
+
+            if (MT.MediaTag && ctx.cache) {
+                MT.MediaTag.setDefaultConfig('Cache', ctx.cache);
+            }
+
             try {
                 var feedback = privateData.feedbackAllowed;
                 Feedback.init(feedback);

www/common/toolbar.js

@@ -229,7 +229,6 @@ MessengerUI, Messages) {
         // Editors
         var pendingFriends = Common.getPendingFriends(); // Friend requests sent
         var friendRequests = Common.getFriendRequests(); // Friend requests received
-        var friendTo = +new Date() - (2 * 24 * 3600 * 1000);
         editUsersNames.forEach(function (data) {
             var name = data.name || Messages.anonymous;
             var $span = $('<span>', {'class': 'cp-avatar'});
@@ -297,7 +296,7 @@ MessengerUI, Messages) {
                 }
             } else if (Common.isLoggedIn() && data.curvePublic && !friends[data.curvePublic]
                 && !priv.readOnly) {
-                if (pendingFriends[data.curvePublic] && pendingFriends[data.curvePublic] > friendTo) {
+                if (pendingFriends[data.curvePublic]) {
                     $('<button>', {
                         'class': 'fa fa-hourglass-half cp-toolbar-userlist-button',
                         'title': Messages.profile_friendRequestSent
@@ -322,7 +321,10 @@ MessengerUI, Messages) {
                     }).appendTo($nameSpan).click(function (e) {
                         e.stopPropagation();
                         Common.sendFriendRequest(data, function (err, obj) {
-                            if (err || (obj && obj.error)) { return void console.error(err || obj.error); }
+                            if (err || (obj && obj.error)) {
+                                UI.warn(Messages.error);
+                                return void console.error(err || obj.error);
+                            }
                         });
                     });
                 }
@@ -338,7 +340,7 @@ MessengerUI, Messages) {
             if (data.profile) {
                 $span.addClass('cp-userlist-clickable');
                 $span.click(function () {
-                    window.open(origin+'/profile/#' + data.profile);
+                    Common.openURL(origin+'/profile/#' + data.profile);
                 });
             }
             Common.displayAvatar($span, data.avatar, name, function () {
@@ -838,10 +840,10 @@ MessengerUI, Messages) {
         var onClick = function (e) {
             e.preventDefault();
             if (e.ctrlKey) {
-                window.open(href);
+                Common.openURL(href);
                 return;
             }
-            window.parent.location = href;
+            Common.gotoURL(href);
         };
 
         var onContext = function (e) { e.stopPropagation(); };
@@ -879,6 +881,14 @@ MessengerUI, Messages) {
                 $spin.text(Messages.saved);
             }, /*local ? 0 :*/ SPINNER_DISAPPEAR_TIME);
         };
+        if (config.spinner) {
+            var h = function () {
+                onSynced();
+                try { config.spinner.onSync.unreg(h); } catch (e) { console.error(e); }
+            };
+            config.spinner.onSync.reg(h);
+            return;
+        }
         config.sfCommon.whenRealtimeSyncs(onSynced);
     };
     var ks = function (toolbar, config, local) {
@@ -891,6 +901,15 @@ MessengerUI, Messages) {
         var $spin = $('<span>', {'class': SPINNER_CLS}).appendTo(toolbar.title);
         $spin.text(Messages.synchronizing);
 
+        if (config.spinner) {
+            config.spinner.onPatch.reg(ks(toolbar, config));
+            typing = 0;
+            setTimeout(function () {
+                kickSpinner(toolbar, config);
+            });
+            return $spin;
+        }
+
         if (config.realtime) {
             config.realtime.onPatch(ks(toolbar, config));
             config.realtime.onMessage(ks(toolbar, config, true));
@@ -990,6 +1009,29 @@ MessengerUI, Messages) {
             h('div.cp-notifications-empty', Messages.notifications_empty)
         ]);
         var pads_options = [div];
+
+        var metadataMgr = config.metadataMgr;
+        var privateData = metadataMgr.getPrivateData();
+        if (!privateData.notifications) {
+            var allowNotif = h('div.cp-notifications-gotoapp', h('p', Messages.allowNotifications));
+            pads_options.unshift(h("hr"));
+            pads_options.unshift(allowNotif);
+            var $allow = $(allowNotif).click(function () {
+                Common.getSframeChannel().event('Q_ASK_NOTIFICATION', null, function (e, allow) {
+                    if (!allow) { return; }
+                    $(allowNotif).remove();
+                });
+            });
+            var onChange = function () {
+                var privateData = metadataMgr.getPrivateData();
+                if (!privateData.notifications) { return; }
+                $allow.remove();
+                metadataMgr.off('change', onChange);
+            };
+            metadataMgr.onChange(onChange);
+        }
+
+
         if (Common.isLoggedIn()) {
             pads_options.unshift(h("hr"));
             pads_options.unshift(openNotifsApp);
@@ -1295,6 +1337,10 @@ MessengerUI, Messages) {
                 toolbar.spinner.text(Messages.reconnecting);
             }
         };
+        toolbar.ready = function () {
+            toolbar.connected = true;
+            kickSpinner(toolbar, config);
+        };
 
         toolbar.errorState = function (state, error) {
             toolbar.isErrorState = state;
@@ -1365,6 +1411,18 @@ MessengerUI, Messages) {
             }
         };
 
+        toolbar.offline = function (bool) {
+            toolbar.connected = !bool; // Can't edit title
+            toolbar.history = bool; // Stop "Initializing" state
+            toolbar.isErrorState = bool; // Stop kickSpinner
+            toolbar.title.toggleClass('cp-toolbar-unsync', bool); // "read only" next to the title
+            if (bool && toolbar.spinner) {
+                toolbar.spinner.text(Messages.offline);
+            } else {
+                kickSpinner(toolbar, config);
+            }
+        };
+
         // On log out, remove permanently the realtime elements of the toolbar
         Common.onLogout(function () {
             failed();

www/common/translations/messages.de.json

@@ -40,7 +40,7 @@
     "saved": "Gespeichert",
     "synced": "Alles gespeichert",
     "deleted": "Gelöscht",
-    "deletedFromServer": "Pad wurde vom Server gelöscht",
+    "deletedFromServer": "Dokument zerstört",
     "mustLogin": "Du musst angemeldet sein, um auf diese Seite zuzugreifen",
     "disabledApp": "Diese Anwendung wurde deaktiviert. Kontaktiere den Administrator dieses CryptPads, um mehr Informationen zu erhalten.",
     "realtime_unrecoverableError": "Es ist ein nicht reparierbarer Fehler aufgetreten. Klicke auf OK, um neu zu laden.",
@@ -547,7 +547,7 @@
     "upload_notEnoughSpace": "Der verfügbare Speicherplatz in deinem CryptDrive reicht leider nicht für diese Datei.",
     "upload_notEnoughSpaceBrief": "Unzureichender Speicherplatz",
     "upload_tooLarge": "Diese Datei überschreitet die für deinen Account erlaubte maximale Größe.",
-    "upload_tooLargeBrief": "Datei zu groß",
+    "upload_tooLargeBrief": "Datei überschreitet die maximale Größe von {0} MB",
     "upload_choose": "Eine Datei wählen",
     "upload_pending": "In der Warteschlange",
     "upload_cancelled": "Abgebrochen",
@@ -1468,5 +1468,33 @@
     "loading_state_1": "Drive laden",
     "loading_state_0": "Oberfläche vorbereiten",
     "loading_state_5": "Dokument rekonstruieren",
-    "error_unhelpfulScriptError": "Skriptfehler: Siehe Konsole im Browser für Details"
+    "error_unhelpfulScriptError": "Skriptfehler: Siehe Konsole im Browser für Details",
+    "documentID": "Kennung des Dokuments",
+    "errorPopupBlocked": "Für die Funktionsweise von CryptPad ist es erforderlich, dass neue Tabs geöffnet werden können. Bitte erlaube Pop-up-Fenster in der Adressleiste deines Browsers. Diese Fenster werden niemals dafür verwendet, dir Werbung anzuzeigen.",
+    "unableToDisplay": "Das Dokument kann nicht angezeigt werden. Drücke ESC, um die Seite neu zu laden. Wenn das Problem weiterhin besteht, kontaktiere bitte den Support.",
+    "mediatag_notReady": "Bitte schließe den Download ab",
+    "settings_mediatagSizeTitle": "Limit für automatisches Herunterladen",
+    "admin_archiveHint": "Ein Dokument unzugänglich machen, ohne es endgültig zu löschen. Es wird in einem Verzeichnis \"archive\" abgelegt und nach einigen Tagen gelöscht (konfigurierbar in der Konfigurationsdatei des Servers).",
+    "mediatag_loadButton": "Anhang laden",
+    "settings_mediatagSizeHint": "Maximale Größe in Megabytes (MB) für das automatische Laden von Medienelementen (Bilder, Videos, PDFs), die in Dokumenten eingebettet sind. Größere Elemente können manuell geladen werden. Gib \"-1\" ein, um Medienelemente immer automatisch zu laden.",
+    "mediatag_saveButton": "Speichern",
+    "admin_archiveInput2": "Passwort für das Dokument",
+    "admin_archiveInput": "URL des Dokuments",
+    "admin_archiveButton": "Archivieren",
+    "Offline": "Offline",
+    "download_zip": "ZIP-Datei erstellen...",
+    "fileTableHeader": "Downloads und Uploads",
+    "allowNotifications": "Benachrichtigungen erlauben",
+    "admin_unarchiveHint": "Ein zuvor archiviertes Dokument wiederherstellen",
+    "pad_mediatagOpen": "Datei öffnen",
+    "pad_mediatagShare": "Datei teilen",
+    "download_zip_file": "Datei {0}/{1}",
+    "archivedFromServer": "Dokument archiviert",
+    "restoredFromServer": "Dokument wiederhergestellt",
+    "admin_archiveInval": "Ungültiges Dokument",
+    "admin_unarchiveButton": "Wiederherstellen",
+    "admin_unarchiveTitle": "Dokumente wiederherstellen",
+    "admin_archiveTitle": "Dokumente archivieren",
+    "history_trimPrompt": "Dieses Dokument hat einen Verlauf von {0} angesammelt, was das Laden verlangsamen kann. Ziehe in Betracht, den Verlauf zu löschen, sofern er nicht benötigt wird.",
+    "contacts_confirmCancel": "Bist du sicher, dass du die Kontaktanfrage an <b>{0}</b> zurücknehmen möchtest?"
 }

www/common/translations/messages.fi.json

@@ -1459,5 +1459,14 @@
     "history_cantRestore": "Palauttaminen epäonnistui. Yhteytesi on katkennut.",
     "history_close": "Sulje",
     "history_restore": "Palauta",
-    "share_bar": "Luo linkki"
+    "share_bar": "Luo linkki",
+    "error_unhelpfulScriptError": "Skriptivirhe: Lisätietoja selaimen kehittäjäkonsolissa",
+    "tag_edit": "Muokkaa",
+    "tag_add": "Lisää",
+    "loading_state_5": "Uudelleenrakenna asiakirja",
+    "loading_state_4": "Lataa Teams",
+    "loading_state_3": "Lataa jaetut kansiot",
+    "loading_state_2": "Päivitä sisältö",
+    "loading_state_1": "Lataa Drive",
+    "loading_state_0": "Rakenna käyttöliittymä"
 }

www/common/translations/messages.fr.json

@@ -41,7 +41,7 @@
     "saved": "Enregistré",
     "synced": "Tout est enregistré",
     "deleted": "Supprimé",
-    "deletedFromServer": "Pad supprimé du serveur",
+    "deletedFromServer": "Document supprimé",
     "mustLogin": "Vous devez être enregistré pour avoir accès à cette page",
     "disabledApp": "Cette application a été désactivée. Pour plus d'information, veuillez contacter l'administrateur de ce CryptPad.",
     "realtime_unrecoverableError": "Une erreur critique est survenue. Cliquez sur OK pour recharger la page.",
@@ -554,7 +554,7 @@
     "upload_notEnoughSpace": "Il n'y a pas assez d'espace libre dans votre CryptDrive pour ce fichier.",
     "upload_notEnoughSpaceBrief": "Pas assez d'espace",
     "upload_tooLarge": "Ce fichier dépasse la taille maximale autorisée pour votre compte.",
-    "upload_tooLargeBrief": "Fichier trop volumineux",
+    "upload_tooLargeBrief": "Le fichier dépasse la limite de {0} Mo",
     "upload_choose": "Choisir un fichier",
     "upload_pending": "En attente",
     "upload_cancelled": "Annulé",
@@ -1468,5 +1468,33 @@
     "loading_state_0": "Construction de l'interface",
     "tag_edit": "Modifier",
     "tag_add": "Ajouter",
-    "error_unhelpfulScriptError": "Erreur de script : consultez la console du navigateur pour plus de détails"
+    "error_unhelpfulScriptError": "Erreur de script : consultez la console du navigateur pour plus de détails",
+    "documentID": "Référence du document",
+    "unableToDisplay": "Impossible d'afficher le document. Veuillez recharger la page avec la touche Échap. Si le problème persiste, veuillez contacter le support.",
+    "errorPopupBlocked": "CryptPad doit pouvoir ouvrir de nouveaux onglets pour fonctionner. Veuillez autoriser les fenêtres pop-up dans la barre d'adresse de votre navigateur. Ces fenêtres ne seront jamais utilisées pour vous montrer de la publicité.",
+    "settings_mediatagSizeHint": "Taille maximale en mégaoctets (Mo) pour le chargement automatique des pièces jointes (images, vidéos, pdf) intégrés dans les documents. Les pièces jointes dont la taille est supérieure à la taille spécifiée peuvent être chargés manuellement. Utilisez \"-1\" pour toujours charger automatiquement les pièces jointes.",
+    "settings_mediatagSizeTitle": "Limite de téléchargement automatique",
+    "mediatag_notReady": "Merci de compléter le téléchargement",
+    "pad_mediatagOpen": "Ouvrir ce fichier",
+    "pad_mediatagShare": "Partager ce fichier",
+    "mediatag_saveButton": "Sauvegarder",
+    "Offline": "Déconnecté",
+    "download_zip_file": "Fichier {0}/{1}",
+    "download_zip": "Construction du fichier ZIP...",
+    "fileTableHeader": "Téléchargements et imports",
+    "allowNotifications": "Autoriser les notifications",
+    "archivedFromServer": "Document archivé",
+    "restoredFromServer": "Document restauré",
+    "admin_archiveInval": "Document invalide",
+    "admin_archiveInput2": "Mot de passe du document",
+    "admin_archiveInput": "URL du document",
+    "admin_unarchiveButton": "Restaurer",
+    "admin_unarchiveHint": "Restaurer un document qui avait été précédemment archivé",
+    "admin_unarchiveTitle": "Restaurer les documents",
+    "admin_archiveButton": "Archiver",
+    "admin_archiveHint": "Rendre un document indisponible sans le supprimer définitivement. Il sera placé dans un répertoire \"archive\" et supprimé après quelques jours (configurable dans le fichier de configuration du serveur).",
+    "admin_archiveTitle": "Archiver les documents",
+    "mediatag_loadButton": "Charger la pièce jointe",
+    "history_trimPrompt": "Ce document a accumulé {0} d'historique qui peut ralentir le temps de chargement. Envisagez de supprimer l'historique s'il n'est pas nécessaire.",
+    "contacts_confirmCancel": "Êtes-vous sûr de vouloir annuler votre demande de contact avec <b>{0}</b> ?"
 }

www/common/translations/messages.ja.json

@@ -249,7 +249,7 @@
     "settings_autostoreMaybe": "手動 (確認しない)",
     "settings_autostoreNo": "手動 (常に確認する)",
     "settings_autostoreHint": "<b>自動</b> あなたがアクセスしたすべてのパッドを、あなたの CryptDrive に保存します。<br><b>手動 (常に確認する)</b> まだ保存していないパッドにアクセスした場合に、あなたの CryptDrive に保存するかどうか尋ねます。<br><b>手動 (確認しない)</b> アクセス先のパッドがあなたの CryptDrive に自動的に保存されなくなります。保存オプションは表示されなくなります。",
-    "settings_userFeedback": "ユーザーフィードバックを有効化",
+    "settings_userFeedback": "ユーザーフィードバックを有効にする",
     "settings_userFeedbackHint2": "あなたのパッドのコンテンツがサーバーと共有されることはありません。",
     "settings_userFeedbackHint1": "CryptPad は、あなたの経験を向上させる方法を知るために、サーバーにいくつかの非常に基本的なフィードバックを提供します。 ",
     "settings_userFeedbackTitle": "フィードバック",
@@ -356,5 +356,72 @@
     "crowdfunding_button": "CryptPad を支援",
     "crowdfunding_home1": "CryptPad はあなたの支援を必要としています！",
     "policy_choices_vpn": "私たちがホストするインスタンスを使用したいが、IP アドレスを私たちに公開したくない場合は、<a href=\"https://www.torproject.org/projects/torbrowser.html.en\" title=\"downloads from the Tor project\" target=\"_blank\" rel=\"noopener noreferrer\">Tor Browser</a> または <a href=\"https://riseup.net/en/vpn\" title=\"VPNs provided by Riseup\" target=\"_blank\" rel=\"noopener noreferrer\">VPN</a> を使用してあなたの IP アドレスを保護できます。",
-    "contacts_removeHistoryTitle": "チャット履歴を削除"
+    "contacts_removeHistoryTitle": "チャット履歴を削除",
+    "properties_passwordSuccessFile": "パスワードは正常に変更されました。",
+    "drive_sfPasswordError": "誤ったパスワードです",
+    "team_title": "チーム: {0}",
+    "password_error": "パッドが存在しません！<br>このエラーは「誤ったパスワードが入力された」場合、または「パッドがサーバーから削除された」場合に発生します。",
+    "password_error_seed": "パッドが存在しません！<br>このエラーは「パスワードが追加・変更された」場合、または「パッドがサーバーから削除された」場合に発生します。",
+    "password_submit": "送信",
+    "password_placeholder": "パスワードを入力...",
+    "password_info": "開こうとしているパッドが存在しないか、パスワードで保護されています。コンテンツにアクセスするには、正しいパスワードを入力してください。",
+    "properties_confirmNew": "パスワードを追加すると、このパッドの URL が変更され、履歴が削除されます。パスワードを知らないユーザーは、このパッドへアクセスできなくなります。続行してよろしいですか？",
+    "properties_changePassword": "パスワードの変更",
+    "password_show": "表示",
+    "properties_addPassword": "パスワードの追加",
+    "history_close": "閉じる",
+    "history_restore": "復元",
+    "fm_emptyTrashOwned": "ごみ箱に、あなたが所有しているドキュメントが入っています。あなたのドライブからのみ<b>削除</b>するか、すべてのユーザーから<b>完全削除</b>するかを選択できます。",
+    "access_destroyPad": "このドキュメントまたはフォルダを完全に削除する",
+    "accessButton": "アクセス",
+    "access_allow": "リスト",
+    "makeACopy": "コピーを作成",
+    "trimHistory_noHistory": "削除可能な履歴がありません",
+    "areYouSure": "クリックして実行",
+    "settings_safeLinksCheckbox": "安全なリンクを有効にする",
+    "settings_safeLinksTitle": "安全なリンク",
+    "settings_safeLinksHint": "CryptPad では、あなたのパッドを解読するための鍵がリンクに含まれています。これは、あなたのブラウザの閲覧履歴にアクセスできる人が、潜在的にあなたの CryptPad のデータを解読・閲覧できることを意味します。この「ブラウザの閲覧履歴にアクセスできる人」には、デバイス間で履歴を同期させる侵入的なブラウザ拡張機能やブラウザが含まれます。「安全なリンク」を有効にすると、鍵がブラウザの閲覧履歴に残ったり、アドレスバーに表示されたりするのを可能な限り防ぐことができます。この機能を有効にした上で共有メニューを使用することを強く推奨します。",
+    "settings_autostoreTitle": "CryptDrive へのパッドの保存",
+    "settings_logoutEverywhereConfirm": "すべてのデバイスでログインが取り消されるため、今後利用する際にもう一度ログインするよう求められます。続行しますか？",
+    "settings_logoutEverywhere": "他のすべてのウェブセッションからログアウトします",
+    "settings_logoutEverywhereTitle": "リモートセッションを閉じる",
+    "loading_state_5": "ドキュメントを再構築",
+    "loading_state_4": "チームを読み込み",
+    "loading_state_3": "共有フォルダを読み込み",
+    "loading_state_2": "コンテンツを更新",
+    "loading_state_1": "ドライブを読み込み",
+    "loading_state_0": "インターフェースを構築",
+    "error_unhelpfulScriptError": "スクリプトエラー: ブラウザコンソールで詳細をご確認ください",
+    "tag_edit": "編集",
+    "tag_add": "追加",
+    "team_exportButton": "ダウンロード",
+    "admin_cat_quota": "ユーザーストレージ",
+    "admin_invalKey": "無効な公開鍵です",
+    "admin_limitPlan": "プラン: {0}",
+    "admin_registrationAllow": "開く",
+    "admin_registrationButton": "閉じる",
+    "oo_version": "バージョン: ",
+    "snapshots_delete": "削除",
+    "snapshots_close": "閉じる",
+    "snapshots_restore": "復元",
+    "snapshots_open": "開く",
+    "snapshots_placeholder": "スナップショット名",
+    "snapshots_new": "新規スナップショット",
+    "snaphot_title": "スナップショット",
+    "snapshots_button": "スナップショット",
+    "filePicker_description": "埋め込むファイルを CryptDrive から選択するか、新規にアップロードしてください",
+    "uploadButtonTitle": "CryptDrive に新規ファイルをアップロード",
+    "uploadFolderButton": "フォルダをアップロード",
+    "uploadButton": "ファイルをアップロード",
+    "filePicker_filter": "ファイル名で検索",
+    "toolbar_theme": "テーマ",
+    "themeButton": "テーマ",
+    "team_cat_back": "チームに戻る",
+    "team_cat_list": "チーム",
+    "allow_checkbox": "アクセスリストを有効にする",
+    "allow_label": "アクセスリスト: {0}",
+    "share_contactCategory": "連絡先",
+    "share_linkCategory": "リンク",
+    "share_linkEdit": "編集",
+    "previewButtonTitle": "マークダウンのプレビューを表示または非表示にします"
 }

www/common/translations/messages.json

@@ -43,7 +43,7 @@
     "saved": "Saved",
     "synced": "Everything is saved",
     "deleted": "Deleted",
-    "deletedFromServer": "Pad deleted from the server",
+    "deletedFromServer": "Document destroyed",
     "mustLogin": "You must be logged in to access this page",
     "disabledApp": "This application has been disabled. Contact the administrator of this CryptPad for more information.",
     "realtime_unrecoverableError": "An unrecoverable error has occured. Click OK to reload.",
@@ -571,7 +571,7 @@
     "upload_notEnoughSpace": "There is not enough space for this file in your CryptDrive.",
     "upload_notEnoughSpaceBrief": "Not enough space",
     "upload_tooLarge": "This file exceeds the maximum upload size allowed for your account.",
-    "upload_tooLargeBrief": "File too large",
+    "upload_tooLargeBrief": "File exceeds the {0}MB limit",
     "upload_choose": "Choose a file",
     "upload_pending": "Pending",
     "upload_cancelled": "Cancelled",
@@ -1468,5 +1468,33 @@
     "loading_state_5": "Reconstruct document",
     "tag_add": "Add",
     "tag_edit": "Edit",
-    "error_unhelpfulScriptError": "Script Error: See browser console for details"
+    "error_unhelpfulScriptError": "Script Error: See browser console for details",
+    "documentID": "Document identifier",
+    "unableToDisplay": "Unable to display the document. Please press Esc to reload the page. If the problem persists, please contact support.",
+    "errorPopupBlocked": "CryptPad needs to be able to open new tabs to operate. Please allow popup windows in your browser's address bar. These windows will never be used to show you advertising.",
+    "admin_archiveTitle": "Archive documents",
+    "admin_archiveHint": "Make a document unavailable without deleting it permanently. It will be placed in an 'archive' directory and deleted after a few days (configurable in the server configuration file).",
+    "admin_archiveButton": "Archive",
+    "admin_unarchiveTitle": "Restore documents",
+    "admin_unarchiveHint": "Restore a document that had previously been archived",
+    "admin_unarchiveButton": "Restore",
+    "admin_archiveInput": "Document URL",
+    "admin_archiveInput2": "Document password",
+    "admin_archiveInval": "Invalid document",
+    "restoredFromServer": "Document restored",
+    "archivedFromServer": "Document archived",
+    "allowNotifications": "Allow notifications",
+    "fileTableHeader": "Downloads and uploads",
+    "download_zip": "Building ZIP file...",
+    "download_zip_file": "File {0}/{1}",
+    "Offline": "Offline",
+    "mediatag_saveButton": "Save",
+    "pad_mediatagShare": "Share file",
+    "pad_mediatagOpen": "Open file",
+    "mediatag_notReady": "Please complete the download",
+    "settings_mediatagSizeTitle": "Automatic download limit",
+    "settings_mediatagSizeHint": "Maximum size in megabytes (MB) for automatically loading media elements (images, videos, pdf) embedded into documents. Elements bigger than the specified size can be loaded manually. Use \"-1\" to always load the media elements automatically.",
+    "mediatag_loadButton": "Load attachment",
+    "history_trimPrompt": "This document has accumulated {0} of history that may slow down loading time. Consider deleting the history if it is not needed.",
+    "contacts_confirmCancel": "Are you sure you want to cancel your contact request with <b>{0}</b>?"
 }

www/debug/main.js

@@ -28,6 +28,8 @@ define([
     // Loaded in load #2
     nThen(function (waitFor) {
         $(waitFor());
+    }).nThen(function (waitFor) {
+        SFCommonO.initIframe(waitFor);
     }).nThen(function (waitFor) {
         var req = {
             cfg: requireConfig,

www/drive/inner.js

@@ -82,6 +82,8 @@ define([
                     var readOnly = !secret.keys.editKeyStr;
                     if (!manager || !manager.folders[fId]) { return; }
                     manager.folders[fId].userObject.setReadOnly(readOnly, secret.keys.secondaryKey);
+
+                    manager.folders[fId].offline = newObj.offline;
                 }));
             });
             // Remove from memory folders that have been deleted from the drive remotely
@@ -310,6 +312,10 @@ define([
                 onReconnect();
             });
             common.onLogout(function () { setEditable(false); });
+
+            // Check if our drive history needs to be trimmed
+            common.checkTrimHistory(null, true);
+
         });
     };
     main();

www/drive/main.js

@@ -54,7 +54,13 @@ define([
                 if (Utils.LocalStore.isLoggedIn()) { return; }
                 Utils.LocalStore.setFSHash('');
                 Utils.LocalStore.clearThumbnail();
-                window.location.reload();
+                try {
+                    Utils.Cache.clear(function () {
+                        window.location.reload();
+                    });
+                } catch (e) {
+                    window.location.reload();
+                }
             });
             sframeChan.on('Q_DRIVE_USEROBJECT', function (data, cb) {
                 Cryptpad.userObjectCommand(data, cb);

www/file/app-file.less

@@ -1,5 +1,6 @@
 @import (reference) '../../customize/src/less2/include/tokenfield.less';
 @import (reference) '../../customize/src/less2/include/framework.less';
+@import (reference) '../../customize/src/less2/include/markdown.less';
 
 &.cp-app-file {
 
@@ -45,6 +46,7 @@
         z-index: -1;
     }
 
+    .mediatag_cryptpad();
     media-tag {
         img {
             max-width: 100%;
@@ -62,7 +64,49 @@
         }
     }
 
-    #cp-app-file-upload-form, #cp-app-file-download-form {
+    #cp-app-file-download-form {
+        padding: 0px;
+        margin: 0px;
+
+        position: relative;
+        display: block;
+        max-width: 90vw;
+        height: 150px;
+        width: ~"min(90vw, 600px)";
+        .cp-app-file-progress-container {
+            margin-top: 5px;
+            height: 40px;
+            font-size: 20px;
+            border: 1px solid @colortheme_logo-2;
+            background: white;
+            color: @cryptpad_text_col;
+            display: flex;
+            justify-content: space-between;
+            position: relative;
+            .cp-app-file-progress-dl {
+                border-right: 1px solid @cryptpad_text_col;
+            }
+            .cp-app-file-progress-dl, .cp-app-file-progress-dc {
+                width: 50%;
+                display: inline-flex;
+                align-items: center;
+                justify-content: center;
+                z-index: 2;
+            }
+            .cp-app-file-progress {
+                z-index: 1;
+                position: absolute;
+                top: 0;
+                left: 0;
+                bottom: 0;
+                background: @colortheme_logo-2;
+            }
+        }
+        .cp-app-file-progress-txt {
+            margin-left: 30px;
+        }
+    }
+    #cp-app-file-upload-form {
         padding: 0px;
         margin: 0px;
 
@@ -154,6 +198,9 @@
                 max-height: 100%;
                 max-width: 100%;
             }
+            &:empty {
+                display: none !important;
+            }
         }
 
     }

www/file/file-crypto.js

@@ -48,69 +48,6 @@ define([
         return new Blob(chunks);
     };
 
-    var concatBuffer = function (a, b) { // TODO make this not so ugly
-        return new Uint8Array(slice(a).concat(slice(b)));
-    };
-
-    var fetchMetadata = function (src, cb) {
-        var done = false;
-        var CB = function (err, res) {
-            if (done) { return; }
-            done = true;
-            cb(err, res);
-        };
-
-        var xhr = new XMLHttpRequest();
-        xhr.open("GET", src, true);
-        xhr.setRequestHeader('Range', 'bytes=0-1');
-        xhr.responseType = 'arraybuffer';
-
-        xhr.onerror= function () { return CB('XHR_ERROR'); };
-        xhr.onload = function () {
-            if (/^4/.test('' + this.status)) { return CB('XHR_ERROR'); }
-            var res = new Uint8Array(xhr.response);
-            var size = decodePrefix(res);
-            var xhr2 = new XMLHttpRequest();
-
-            xhr2.open("GET", src, true);
-            xhr2.setRequestHeader('Range', 'bytes=2-' + (size + 2));
-            xhr2.responseType = 'arraybuffer';
-            xhr2.onload = function () {
-                if (/^4/.test('' + this.status)) { return CB('XHR_ERROR'); }
-                var res2 = new Uint8Array(xhr2.response);
-                var all = concatBuffer(res, res2);
-                CB(void 0, all);
-            };
-            xhr2.send(null);
-        };
-        xhr.send(null);
-    };
-
-    var decryptMetadata = function (u8, key) {
-        var prefix = u8.subarray(0, 2);
-        var metadataLength = decodePrefix(prefix);
-
-        var metaBox = new Uint8Array(u8.subarray(2, 2 + metadataLength));
-        var metaChunk = Nacl.secretbox.open(metaBox, createNonce(), key);
-
-        try {
-            return JSON.parse(Nacl.util.encodeUTF8(metaChunk));
-        }
-        catch (e) { return null; }
-    };
-
-    var fetchDecryptedMetadata = function (src, key, cb) {
-        if (typeof(src) !== 'string') {
-            return window.setTimeout(function () {
-                cb('NO_SOURCE');
-            });
-        }
-        fetchMetadata(src, function (e, buffer) {
-            if (e) { return cb(e); }
-            cb(void 0, decryptMetadata(buffer, key));
-        });
-    };
-
     var decrypt = function (u8, key, done, progress) {
         var MAX = u8.length;
         var _progress = function (offset) {
@@ -128,6 +65,11 @@ define([
             metadata: undefined,
         };
 
+        var cancelled = false;
+        var cancel = function () {
+            cancelled = true;
+        };
+
         var metaBox = new Uint8Array(u8.subarray(2, 2 + metadataLength));
 
         var metaChunk = Nacl.secretbox.open(metaBox, nonce, key);
@@ -168,6 +110,7 @@ define([
         var chunks = [];
 
         var again = function () {
+            if (cancelled) { return; }
             takeChunk(function (e, plaintext) {
                 if (e) {
                     return setTimeout(function () {
@@ -188,6 +131,10 @@ define([
         };
 
         again();
+
+        return {
+            cancel: cancel
+        };
     };
 
     // metadata
@@ -258,8 +205,5 @@ define([
         encrypt: encrypt,
         joinChunks: joinChunks,
         computeEncryptedSize: computeEncryptedSize,
-        decryptMetadata: decryptMetadata,
-        fetchMetadata: fetchMetadata,
-        fetchDecryptedMetadata: fetchDecryptedMetadata,
     };
 });

www/file/inner.html

@@ -16,11 +16,6 @@
             <label for="cp-app-file-upfile" class="btn btn-primary cp-app-file-block unselectable" data-localization-title="upload_choose"
                 data-localization="upload_choose"></label>
         </div>
-        <div id="cp-app-file-download-form" style="display: none;">
-            <input type="button" name="dl" id="cp-app-file-dlfile" class="cp-app-file-input" />
-            <label for="cp-app-file-dlfile" class="btn btn-success cp-app-file-block unselectable" data-localization-title="download_button"><span data-localization="download_button"></span></label>
-            <span class="cp-app-file-block" id="cp-app-file-dlprogress"></span>
-        </div>
         <div id="cp-app-file-download-view" style="display: none;">
             <media-tag id="cp-app-file-view"></media-tag>
         </div>

www/file/inner.js

@@ -8,6 +8,7 @@ define([
     '/common/common-util.js',
     '/common/common-hash.js',
     '/common/common-interface.js',
+    '/common/hyperscript.js',
     '/customize/messages.js',
 
     '/file/file-crypto.js',
@@ -29,6 +30,7 @@ define([
     Util,
     Hash,
     UI,
+    h,
     Messages,
     FileCrypto,
     MediaTag)
@@ -37,18 +39,12 @@ define([
     var Nacl = window.nacl;
 
     var APP = window.APP = {};
-    MediaTag.setDefaultConfig('download', {
-        text: Messages.download_mt_button
-    });
 
     var andThen = function (common) {
         var $appContainer = $('#cp-app-file-content');
         var $form = $('#cp-app-file-upload-form');
-        var $dlform = $('#cp-app-file-download-form');
         var $dlview = $('#cp-app-file-download-view');
         var $label = $form.find('label');
-        var $dllabel = $dlform.find('label span');
-        var $progress = $('#cp-app-file-dlprogress');
         var $bar = $('.cp-toolbar-container');
         var $body = $('body');
 
@@ -88,142 +84,86 @@ define([
         var toolbar = APP.toolbar = Toolbar.create(configTb);
 
         if (!uploadMode) {
-            var hexFileName = secret.channel;
+            (function () {
-            var src = fileHost + Hash.getBlobPathFromHex(hexFileName);
+                var hexFileName = secret.channel;
-            var key = secret.keys && secret.keys.cryptKey;
+                var src = fileHost + Hash.getBlobPathFromHex(hexFileName);
-            var cryptKey = Nacl.util.encodeBase64(key);
+                var key = secret.keys && secret.keys.cryptKey;
-
+                var cryptKey = Nacl.util.encodeBase64(key);
-            FileCrypto.fetchDecryptedMetadata(src, key, function (e, metadata) {
+
-                if (e) {
+                var $mt = $dlview.find('media-tag');
-                    if (e === 'XHR_ERROR') {
+                $mt.attr('src', src);
-                        return void UI.errorLoadingScreen(Messages.download_resourceNotAvailable, false, function () {
+                $mt.attr('data-crypto-key', 'cryptpad:'+cryptKey);
-                            common.gotoURL('/file/');
+                $mt.css('transform', 'scale(2)');
-                        });
+
+                var rightsideDisplayed = false;
+                var metadataReceived = false;
+                UI.removeLoadingScreen();
+                $dlview.show();
+
+                MediaTag($mt[0]).on('complete', function (decrypted) {
+                    $mt.css('transform', '');
+                    if (!rightsideDisplayed) {
+                        toolbar.$drawer
+                        .append(common.createButton('export', true, {}, function () {
+                            saveAs(decrypted.content, decrypted.metadata.name);
+                        }));
+                        rightsideDisplayed = true;
                     }
-                    return void console.error(e);
-                }
-
-                // Add pad attributes when the file is saved in the drive
-                Title.onTitleChange(function () {
-                    var owners = metadata.owners;
-                    if (owners) { common.setPadAttribute('owners', owners); }
-                    common.setPadAttribute('fileType', metadata.type);
-                });
-                $(document).on('cpPadStored', function () {
-                    var owners = metadata.owners;
-                    if (owners) { common.setPadAttribute('owners', owners); }
-                    common.setPadAttribute('fileType', metadata.type);
-                });
-
-                // Save to the drive or update the acces time
-                var title = document.title = metadata.name;
-                Title.updateTitle(title || Title.defaultTitle);
-
-                var owners = metadata.owners;
-                if (owners) {
-                    common.setPadAttribute('owners', owners);
-                }
-                if (metadata.type) {
-                    common.setPadAttribute('fileType', metadata.type);
-                }
-
-                toolbar.addElement(['pageTitle'], {
-                    pageTitle: title,
-                    title: Title.getTitleConfig(),
-                });
-                toolbar.$drawer.append(common.createButton('forget', true));
-                toolbar.$drawer.append(common.createButton('properties', true));
-                if (common.isLoggedIn()) {
-                    toolbar.$drawer.append(common.createButton('hashtag', true));
-                }
-                toolbar.$file.show();
-
-                var displayFile = function (ev, sizeMb, CB) {
-                    var called_back;
-                    var cb = function (e) {
-                        if (called_back) { return; }
-                        called_back = true;
-                        if (CB) { CB(e); }
-                    };
-
-                    var $mt = $dlview.find('media-tag');
-                    $mt.attr('src', src);
-                    $mt.attr('data-crypto-key', 'cryptpad:'+cryptKey);
 
-                    var rightsideDisplayed = false;
+                    // make pdfs big
-
+                    var toolbarHeight = $('#cp-toolbar').height();
-                    MediaTag($mt[0]).on('complete', function (decrypted) {
+                    $('media-tag iframe').css({
-                        $dlview.show();
+                        'height': 'calc(100vh - ' + toolbarHeight + 'px)',
-                        $dlform.hide();
+                        'width': '100vw',
-                        var $dlButton = $dlview.find('media-tag button');
+                        'position': 'absolute',
-                        if (ev) { $dlButton.click(); }
+                        'bottom': 0,
-
+                        'left': 0,
-                        if (!rightsideDisplayed) {
+                        'border': 0
-                            toolbar.$drawer
+                    });
-                            .append(common.createButton('export', true, {}, function () {
+                }).on('metadata', function (metadata) {
-                                saveAs(decrypted.content, decrypted.metadata.name);
+                    if (metadataReceived) { return; }
-                            }));
+                    metadataReceived = true;
-                            rightsideDisplayed = true;
+                    // Add pad attributes when the file is saved in the drive
-                        }
+                    Title.onTitleChange(function () {
-
+                        var owners = metadata.owners;
-                        // make pdfs big
+                        if (owners) { common.setPadAttribute('owners', owners); }
-                        var toolbarHeight = $('#cp-toolbar').height();
+                        common.setPadAttribute('fileType', metadata.type);
-                        var $another_iframe = $('media-tag iframe').css({
+                    });
-                            'height': 'calc(100vh - ' + toolbarHeight + 'px)',
+                    $(document).on('cpPadStored', function () {
-                            'width': '100vw',
+                        var owners = metadata.owners;
-                            'position': 'absolute',
+                        if (owners) { common.setPadAttribute('owners', owners); }
-                            'bottom': 0,
+                        common.setPadAttribute('fileType', metadata.type);
-                            'left': 0,
-                            'border': 0
-                        });
-
-                        if ($another_iframe.length) {
-                            $another_iframe.load(function () {
-                                cb();
-                            });
-                        } else {
-                            cb();
-                        }
-                    }).on('progress', function (data) {
-                        var p = data.progress +'%';
-                        $progress.width(p);
-                    }).on('error', function (err) {
-                        console.error(err);
                     });
-                };
 
-                var todoBigFile = function (sizeMb) {
+                    // Save to the drive or update the acces time
-                    $dlform.show();
+                    var title = document.title = metadata.name;
-                    UI.removeLoadingScreen();
+                    Title.updateTitle(title || Title.defaultTitle);
-                    $dllabel.append($('<br>'));
-                    $dllabel.append(Util.fixHTML(metadata.name));
 
-                    // don't display the size if you don't know it.
+                    var owners = metadata.owners;
-                    if (typeof(sizeM) === 'number') {
+                    if (owners) {
-                        $dllabel.append($('<br>'));
+                        common.setPadAttribute('owners', owners);
-                        $dllabel.append(Messages._getKey('formattedMB', [sizeMb]));
                     }
-                    var decrypting = false;
+                    if (metadata.type) {
-                    var onClick = function (ev) {
+                        common.setPadAttribute('fileType', metadata.type);
-                        if (decrypting) { return; }
-                        decrypting = true;
-                        displayFile(ev, sizeMb, function (err) {
-                            $appContainer.css('background-color',
-                                              common.getAppConfig().appBackgroundColor);
-                            if (err) { UI.alert(err); }
-                        });
-                    };
-                    if (typeof(sizeMb) === 'number' && sizeMb < 5) { return void onClick(); }
-                    $dlform.find('#cp-app-file-dlfile, #cp-app-file-dlprogress').click(onClick);
-                };
-                common.getFileSize(hexFileName, function (e, data) {
-                    if (e) {
-                        return void UI.errorLoadingScreen(e);
                     }
-                    var size = Util.bytesToMegabytes(data);
+
-                    return void todoBigFile(size);
+                    toolbar.addElement(['pageTitle'], {
+                        pageTitle: title,
+                        title: Title.getTitleConfig(),
+                    });
+                    toolbar.$drawer.append(common.createButton('forget', true));
+                    toolbar.$drawer.append(common.createButton('properties', true));
+                    if (common.isLoggedIn()) {
+                        toolbar.$drawer.append(common.createButton('hashtag', true));
+                    }
+                    toolbar.$file.show();
+                }).on('error', function (err) {
+                    $appContainer.css('background-color',
+                                      common.getAppConfig().appBackgroundColor);
+                    UI.warn(Messages.error);
+                    console.error(err);
                 });
-            });
+            })();
             return;
         }
 

www/oodoc/inner.html

@@ -9,9 +9,11 @@
 </head>
 <body class="cp-app-oodoc">
     <div id="cp-toolbar" class="cp-toolbar-container"></div>
-    <div id="cp-app-oo-editor">
+    <div id="cp-app-oo-container">
-        <div id="cp-app-oo-placeholder"></div>
+        <div id="cp-app-oo-editor">
-        <script type="text/javascript" src="/common/onlyoffice/web-apps/apps/api/documents/api.js"></script>
+            <div id="cp-app-oo-placeholder-a"></div>
+            <!--<script type="text/javascript" src="/common/onlyoffice/web-apps/apps/api/documents/api.js"></script>-->
+        </div>
     </div>
 </body>
 

www/ooslide/inner.html

@@ -9,9 +9,11 @@
 </head>
 <body class="cp-app-ooslide">
     <div id="cp-toolbar" class="cp-toolbar-container"></div>
-    <div id="cp-app-oo-editor">
+    <div id="cp-app-oo-container">
-        <div id="cp-app-oo-placeholder"></div>
+        <div id="cp-app-oo-editor">
-        <script type="text/javascript" src="/common/onlyoffice/web-apps/apps/api/documents/api.js"></script>
+            <div id="cp-app-oo-placeholder-a"></div>
+            <!--<script type="text/javascript" src="/common/onlyoffice/web-apps/apps/api/documents/api.js"></script>-->
+        </div>
     </div>
 </body>
 

www/pad/inner.js

@@ -46,6 +46,7 @@ define([
     '/common/test.js',
 
     '/bower_components/diff-dom/diffDOM.js',
+    '/bower_components/file-saver/FileSaver.min.js',
 
     'css!/customize/src/print.css',
     'css!/bower_components/bootstrap/dist/css/bootstrap.min.css',
@@ -462,7 +463,9 @@ define([
         setTimeout(function() { // Just in case
             var tags = dom.querySelectorAll('media-tag:empty');
             Array.prototype.slice.call(tags).forEach(function(el) {
-                MediaTag(el);
+                var mediaObject = MediaTag(el, {
+                    body: dom
+                });
                 $(el).on('keydown', function(e) {
                     if ([8, 46].indexOf(e.which) !== -1) {
                         $(el).remove();
@@ -472,13 +475,17 @@ define([
                 var observer = new MutationObserver(function(mutations) {
                     mutations.forEach(function(mutation) {
                         if (mutation.type === 'childList') {
-                            var list_values = [].slice.call(el.children);
+                            var list_values = slice(el.children)
+                                                .map(function (el) { return el.outerHTML; })
+                                                .join('');
                             mediaTagMap[el.getAttribute('src')] = list_values;
+                            if (mediaObject.complete) { observer.disconnect(); }
                         }
                     });
                 });
                 observer.observe(el, {
                     attributes: false,
+                    subtree: true,
                     childList: true,
                     characterData: false
                 });
@@ -491,9 +498,10 @@ define([
         Array.prototype.slice.call(tags).forEach(function(tag) {
             var src = tag.getAttribute('src');
             if (mediaTagMap[src]) {
-                mediaTagMap[src].forEach(function(n) {
+                tag.innerHTML = mediaTagMap[src];
-                    tag.appendChild(n.cloneNode());
+                /*mediaTagMap[src].forEach(function(n) {
-                });
+                    tag.appendChild(n.cloneNode(true));
+                });*/
             }
         });
     };
@@ -1084,6 +1092,9 @@ define([
                     border: Messages.pad_mediatagBorder,
                     preview: Messages.pad_mediatagPreview,
                     'import': Messages.pad_mediatagImport,
+                    download: Messages.download_mt_button,
+                    share: Messages.pad_mediatagShare,
+                    open: Messages.pad_mediatagOpen,
                     options: Messages.pad_mediatagOptions
                 };
                 Ckeditor._commentsTranslations = {
@@ -1164,6 +1175,28 @@ define([
                 editor.plugins.mediatag.import = function($mt) {
                     framework._.sfCommon.importMediaTag($mt);
                 };
+                editor.plugins.mediatag.download = function($mt) {
+                    var media = Util.find($mt, [0, '_mediaObject']);
+                    if (!media) { return void console.error('no media'); }
+                    if (!media.complete) { return void UI.warn(Messages.mediatag_notReady); }
+                    if (!(media && media._blob)) { return void console.error($mt); }
+                    window.saveAs(media._blob.content, media.name);
+                };
+                editor.plugins.mediatag.open = function($mt) {
+                    var hash = framework._.sfCommon.getHashFromMediaTag($mt);
+                    framework._.sfCommon.openURL(Hash.hashToHref(hash, 'file'));
+                };
+                editor.plugins.mediatag.share = function($mt) {
+                    var data = {
+                        file: true,
+                        pathname: '/file/',
+                        hashes: {
+                            fileHash: framework._.sfCommon.getHashFromMediaTag($mt)
+                        },
+                        title: Util.find($mt[0], ['_mediaObject', 'name']) || ''
+                    };
+                    framework._.sfCommon.getSframeChannel().event('EV_SHARE_OPEN', data);
+                };
                 Links.init(Ckeditor, editor);
             }).nThen(function() {
                 // Move ckeditor parts to have a structure like the other apps

www/pad/mediatag-plugin.js

@@ -53,15 +53,57 @@
                     editor.plugins.mediatag.import($mt);
                 }
             });
+            editor.addCommand('downloadMT', {
+                exec: function (editor) {
+                    var w = targetWidget;
+                    targetWidget = undefined;
+                    var $mt = $(w.$).find('media-tag');
+                    editor.plugins.mediatag.download($mt);
+                }
+            });
+            editor.addCommand('openMT', {
+                exec: function (editor) {
+                    var w = targetWidget;
+                    targetWidget = undefined;
+                    var $mt = $(w.$).find('media-tag');
+                    editor.plugins.mediatag.open($mt);
+                }
+            });
+            editor.addCommand('shareMT', {
+                exec: function (editor) {
+                    var w = targetWidget;
+                    targetWidget = undefined;
+                    var $mt = $(w.$).find('media-tag');
+                    editor.plugins.mediatag.share($mt);
+                }
+            });
 
             if (editor.addMenuItems) {
                 editor.addMenuGroup('mediatag');
+                editor.addMenuItem('open', {
+                        label: Messages.open,
+                        icon: 'iframe',
+                        command: 'openMT',
+                        group: 'mediatag'
+                });
+                editor.addMenuItem('share', {
+                        label: Messages.share,
+                        icon: 'link',
+                        command: 'shareMT',
+                        group: 'mediatag'
+                });
                 editor.addMenuItem('importMediatag', {
                         label: Messages.import,
                         icon: 'save',
                         command: 'importMediatag',
                         group: 'mediatag'
                 });
+                editor.addMenuItem('download', {
+                        label: Messages.download,
+                        icon: 'save',
+                        command: 'downloadMT',
+                        group: 'mediatag'
+                });
                 editor.addMenuItem('mediatag', {
                         label: Messages.options,
                         icon: 'image',
@@ -76,6 +118,9 @@
                         targetWidget = element;
                         return {
                             mediatag: CKEDITOR.TRISTATE_OFF,
+                            open: CKEDITOR.TRISTATE_OFF,
+                            share: CKEDITOR.TRISTATE_OFF,
+                            download: CKEDITOR.TRISTATE_OFF,
                             importMediatag: CKEDITOR.TRISTATE_OFF,
                         };
                     }

www/poll/inner.js

@@ -1087,6 +1087,8 @@ define([
         common.openPadChat(function () {});
 
         UI.removeLoadingScreen();
+
+        common.checkTrimHistory();
     };
 
     var onError = function (info) {

www/profile/inner.js

@@ -101,13 +101,13 @@ define([
         var url = APP.origin + '/profile/#' + hash;
 
         $('<button>', {
-            'class': 'btn btn-success '+VIEW_PROFILE_BUTTON,
+            'class': 'btn '+VIEW_PROFILE_BUTTON,
         }).text(Messages.profile_viewMyProfile).click(function () {
             window.open(url, '_blank');
         }).appendTo($container);
 
         $('<button>', {
-            'class': 'btn btn-success '+VIEW_PROFILE_BUTTON,
+            'class': 'btn btn-primary '+VIEW_PROFILE_BUTTON,
         }).append(h('i.fa.fa-shhare-alt'))
           .append(h('span', Messages.shareButton))
           .click(function () {
@@ -136,7 +136,7 @@ define([
         APP.$linkEdit = $();
         if (APP.readOnly) { return; }
 
-        var button = h('button.btn.btn-primary', {
+        var button = h('button.btn', {
             title: Messages.clickToEdit
         }, Messages.profile_addLink);
         APP.$linkEdit = $(button);
@@ -243,10 +243,34 @@ define([
             return;
         }
 
+        var addCancel = function () {
+            var cancelButton = h('button.btn.btn-danger.cp-app-profile-friend-request', [
+                h('i.fa.fa-user-times'),
+                Messages.cancel
+            ]);
+            $(cancelButton).click(function () {
+                // Unfriend confirm
+                var content = h('div', [
+                    UI.setHTML(h('p'), Messages._getKey('contacts_confirmCancel', [name]))
+                ]);
+                UI.confirm(content, function (yes) {
+                    if (!yes) { return; }
+                    module.execCommand('CANCEL_FRIEND', {
+                        curvePublic: data.curvePublic,
+                        notifications: data.notifications
+                    }, function (e) {
+                        refreshFriendRequest(data);
+                        if (e) { UI.warn(Messages.error); return void console.error(e); }
+                    });
+                });
+            }).appendTo(APP.$friend);
+        };
+
         // Pending friend (we've sent a friend request)
         var pendingFriends = APP.common.getPendingFriends(); // Friend requests sent
         if (pendingFriends[data.curvePublic]) {
             $button.attr('disabled', 'disabled').append(Messages.profile_friendRequestSent);
+            addCancel();
             return;
         }
         // This is not a friend yet: we can send a friend request
@@ -255,8 +279,9 @@ define([
                 APP.common.sendFriendRequest({
                     curvePublic: data.curvePublic,
                     notifications: data.notifications
-                }, function () {
+                }, function (err, obj) {
-                    $button.attr('disabled', 'disabled').append(Messages.profile_friendRequestSent);
+                    if (obj && obj.error) { return void UI.warn(Messages.error); }
+                    //$button.attr('disabled', 'disabled').append(Messages.profile_friendRequestSent);
                 });
             });
     };
@@ -468,7 +493,7 @@ define([
 
         var $div = $(h('div.cp-sidebarlayout-element')).appendTo($container);
         APP.$edPublic = $('<button>', {
-            'class': 'btn btn-success',
+            'class': 'btn',
         }).append(h('i.fa.fa-key'))
           .append(h('span', Messages.profile_copyKey))
           .click(function () {

www/profile/main.js

@@ -50,7 +50,7 @@ define([
                 cb(null, secret);
             });
         };
-        var addData = function (meta, Cryptad, user) {
+        var addData = function (meta, Cryptpad, user) {
             meta.isOwnProfile = !window.location.hash ||
                 window.location.hash.slice(1) === user.profile;
         };

www/secureiframe/inner.js

@@ -52,10 +52,10 @@ define([
                                               : Share.getShareModal;
                 f(common, {
                     origin: priv.origin,
-                    pathname: priv.pathname,
+                    pathname: data.pathname || priv.pathname,
-                    password: priv.password,
+                    password: data.hashes ? '' : priv.password,
-                    isTemplate: priv.isTemplate,
+                    isTemplate: data.hashes ? false : priv.isTemplate,
-                    hashes: priv.hashes,
+                    hashes: data.hashes || priv.hashes,
                     common: common,
                     title: data.title,
                     versionHash: data.versionHash,
@@ -64,8 +64,8 @@ define([
                         hideIframe();
                     },
                     fileData: {
-                        hash: priv.hashes.fileHash,
+                        hash: (data.hashes && data.hashes.fileHash) || priv.hashes.fileHash,
-                        password: priv.password
+                        password: data.hashes ? '' :  priv.password
                     }
                 }, function (e, modal) {
                     if (e) { console.error(e); }

www/secureiframe/main.js

@@ -33,7 +33,7 @@ define([
             // loading screen setup.
             var done = waitFor();
             var onMsg = function (msg) {
-                var data = JSON.parse(msg.data);
+                var data = typeof(msg.data) === "object" ? msg.data : JSON.parse(msg.data);
                 if (data.q !== 'READY') { return; }
                 window.removeEventListener('message', onMsg);
                 var _done = done;

www/settings/app-settings.less

@@ -70,6 +70,10 @@
                         margin-right: 100%;
                     }
                 }
+                & > .fa {
+                    align-self: center;
+                    margin-right: -16px;
+                }
             }
             .cp-settings-info-block {
                 [type="text"] {

www/settings/inner.js

@@ -51,7 +51,7 @@ define([
             'cp-settings-info-block',
             'cp-settings-displayname',
             'cp-settings-language-selector',
-            'cp-settings-resettips',
+            'cp-settings-mediatag-size',
             'cp-settings-change-password',
             'cp-settings-delete'
         ],
@@ -62,6 +62,7 @@ define([
             'cp-settings-userfeedback',
         ],
         'drive': [
+            'cp-settings-resettips',
             'cp-settings-drive-duplicate',
             'cp-settings-thumbnails',
             'cp-settings-drive-backup',
@@ -576,6 +577,58 @@ define([
         cb(form);
     }, true);
 
+    makeBlock('mediatag-size', function(cb) {
+        var $inputBlock = $('<div>', {
+            'class': 'cp-sidebarlayout-input-block',
+        });
+
+        var spinner;
+        var $input = $('<input>', {
+            'min': -1,
+            'max': 1000,
+            type: 'number',
+        }).appendTo($inputBlock);
+
+        var oldVal;
+
+        var todo = function () {
+            var val = parseInt($input.val());
+            if (typeof(val) !== 'number' || isNaN(val)) { return UI.warn(Messages.error); }
+            if (val === oldVal) { return; }
+            spinner.spin();
+            common.setAttribute(['general', 'mediatag-size'], val, function (err) {
+                if (err) {
+                    spinner.hide();
+                    console.error(err);
+                    return UI.warn(Messages.error);
+                }
+                oldVal = val;
+                spinner.done();
+                UI.log(Messages.saved);
+            });
+        };
+        var $save = $(h('button.btn.btn-primary', Messages.settings_save)).appendTo($inputBlock);
+        spinner = UI.makeSpinner($inputBlock);
+
+        $save.click(todo);
+        $input.on('keyup', function(e) {
+            if (e.which === 13) { todo(); }
+        });
+
+        common.getAttribute(['general', 'mediatag-size'], function(e, val) {
+            if (e) { return void console.error(e); }
+            if (typeof(val) !== 'number' || isNaN(val)) {
+                oldVal = 5;
+                $input.val(5);
+            } else {
+                oldVal = val;
+                $input.val(val);
+            }
+        });
+
+        cb($inputBlock);
+    }, true);
+
     // Security
 
     makeBlock('safe-links', function(cb) {
@@ -777,7 +830,7 @@ define([
                         Feedback.send('FULL_DRIVE_EXPORT_COMPLETE');
                         saveAs(blob, filename);
                     }, errors);
-                }, ui.update);
+                }, ui.update, common.getCache());
                 ui.onCancel(function() {
                     ui.close();
                     bu.stop();
@@ -903,7 +956,7 @@ define([
         cb(content);
     };
     makeBlock('trim-history', function(cb, $div) {
-        if (!common.isLoggedIn()) { return; }
+        if (!common.isLoggedIn()) { return void cb(false); }
         redrawTrimHistory(cb, $div);
     }, true);
 

www/support/ui.js

@@ -230,7 +230,7 @@ define([
         var form = h('div.cp-support-form-container', content);
 
         $(cancel).click(function () {
-            $(form).closest('.cp-support-list-ticket').find('.cp-support-list-actions').show();
+            $(form).closest('.cp-support-list-ticket').find('.cp-support-list-actions').css('display', '');
             $(form).remove();
         });
 
@@ -257,8 +257,9 @@ define([
         var url;
         if (ctx.isAdmin) {
             ticketCategory = Messages['support_cat_'+(content.category || 'all')] + ' - ';
-            url = h('button.btn.btn-primary.fa.fa-clipboard');
+            url = h('button.btn.fa.fa-clipboard');
-            $(url).click(function () {
+            $(url).click(function (e) {
+                e.stopPropagation();
                 var link = privateData.origin + privateData.pathname + '#' + 'support-' + content.id;
                 var success = Clipboard.copy(link);
                 if (success) { UI.log(Messages.shareSuccess); }
@@ -269,7 +270,10 @@ define([
             'data-cat': content.category,
             'data-id': content.id
         }, [
-            h('h2', [ticketCategory, ticketTitle, url]),
+            h('h2', [
+                h('span', [ticketCategory, ticketTitle]),
+                h('span.cp-support-title-buttons',url)
+            ]),
             actions
         ]));
 
@@ -303,7 +307,7 @@ define([
             var form = makeForm(ctx, function () {
                 var sent = sendForm(ctx, content.id, form, content.sender);
                 if (sent) {
-                    $(actions).show();
+                    $(actions).css('display', '');
                     $(form).remove();
                 }
             }, content.title);
@@ -397,6 +401,7 @@ define([
 
         var fmConfig = {
             body: $('body'),
+            noStore: true, // Don't store attachments into our drive
             onUploaded: function (ev, data) {
                 if (ev.callback) {
                     ev.callback(data);

www/teams/inner.js

@@ -46,7 +46,9 @@ define([
     Backup,
     Messages)
 {
-    var APP = {};
+    var APP = {
+        teams: {}
+    };
     var driveAPP = {};
     var saveAs = window.saveAs;
     //var SHARED_FOLDER_NAME = Messages.fm_sharedFolderName;
@@ -91,6 +93,8 @@ define([
                     var readOnly = !secret.keys.editKeyStr;
                     if (!manager || !manager.folders[fId]) { return; }
                     manager.folders[fId].userObject.setReadOnly(readOnly, secret.keys.secondaryKey);
+
+                    manager.folders[fId].offline = newObj.offline;
                 }));
             });
             // Remove from memory folders that have been deleted from the drive remotely
@@ -211,6 +215,11 @@ define([
                 if (obj && obj.error) {
                     return void UI.warn(Messages.error);
                 }
+
+                // Refresh offline state
+                APP.teams[APP.team] = APP.teams[APP.team] || {};
+                APP.teams[APP.team].offline = obj.offline;
+
                 common.displayAvatar($avatar, obj.avatar, obj.name);
                 $category.append($avatar);
                 $avatar.append(h('span.cp-sidebarlayout-category-name', obj.name));
@@ -333,6 +342,11 @@ define([
             });
             APP.drive = drive;
             driveAPP.refresh = drive.refresh;
+
+            if (APP.teams[id] && APP.teams[id].offline) {
+                setEditable(false);
+                drive.refresh();
+            }
         });
     };
 
@@ -406,7 +420,15 @@ define([
 
             content.push(h('h3', Messages.team_listTitle + ' ' + slots));
 
+            APP.teams = {};
+
             keys.forEach(function (id) {
+                if (!obj[id].empty) {
+                    APP.teams[id] = {
+                        offline: obj[id] && obj[id].offline
+                    };
+                }
+
                 var team = obj[id];
                 if (team.empty) {
                     list.push(h('div.cp-team-list-team.empty', [
@@ -454,7 +476,7 @@ define([
         var getWarningBox = function () {
             return h('div.alert.alert-warning', {
                 role:'alert'
-            }, isOwner ? Messages.team_maxOwner : Messages._getKey('team_maxTeams', [MAX_TEAMS_SLOTS]));
+            }, Messages._getKey('team_maxTeams', [MAX_TEAMS_SLOTS]));
         };
 
         if (Object.keys(privateData.teams || {}).length >= Constants.MAX_TEAMS_SLOTS || isOwner) {
@@ -1049,7 +1071,7 @@ define([
                         Feedback.send('FULL_TEAMDRIVE_EXPORT_COMPLETE');
                         saveAs(blob, filename);
                     }, errors);
-                }, ui.update);
+                }, ui.update, common.getCache);
                 ui.onCancel(function() {
                     ui.close();
                     bu.stop();
@@ -1433,13 +1455,15 @@ define([
                 }
             });
 
-            var onDisconnect = function (noAlert) {
+            var onDisconnect = function (teamId) {
+                if (APP.team && teamId && APP.team !== teamId) { return; }
                 setEditable(false);
                 if (APP.team && driveAPP.refresh) { driveAPP.refresh(); }
                 toolbar.failed();
-                if (!noAlert) { UIElements.disconnectAlert(); }
+                UIElements.disconnectAlert();
             };
-            var onReconnect = function () {
+            var onReconnect = function (teamId) {
+                if (APP.team && teamId && APP.team !== teamId) { return; }
                 setEditable(true);
                 if (APP.team && driveAPP.refresh) { driveAPP.refresh(); }
                 toolbar.reconnecting();
@@ -1449,11 +1473,17 @@ define([
             sframeChan.on('EV_DRIVE_LOG', function (msg) {
                 UI.log(msg);
             });
-            sframeChan.on('EV_NETWORK_DISCONNECT', function () {
+            sframeChan.on('EV_NETWORK_DISCONNECT', function (teamId) {
-                onDisconnect();
+                onDisconnect(teamId);
+                if (teamId && APP.teams[teamId]) {
+                    APP.teams[teamId].offline = true;
+                }
             });
-            sframeChan.on('EV_NETWORK_RECONNECT', function () {
+            sframeChan.on('EV_NETWORK_RECONNECT', function (teamId) {
-                onReconnect();
+                onReconnect(teamId);
+                if (teamId && APP.teams[teamId]) {
+                    APP.teams[teamId].offline = false;
+                }
             });
             common.onLogout(function () { setEditable(false); });
         });

www/teams/main.js

@@ -55,10 +55,10 @@ define([
                     sframeChan.event('EV_'+obj.data.ev, obj.data.data);
                 }
                 if (obj.data.ev === 'NETWORK_RECONNECT') {
-                    sframeChan.event('EV_NETWORK_RECONNECT');
+                    sframeChan.event('EV_NETWORK_RECONNECT', obj.data.data);
                 }
                 if (obj.data.ev === 'NETWORK_DISCONNECT') {
-                    sframeChan.event('EV_NETWORK_DISCONNECT');
+                    sframeChan.event('EV_NETWORK_DISCONNECT', obj.data.data);
                 }
             });