From a87d0f294398814129a00b4c3813d33f87230c14 Mon Sep 17 00:00:00 2001 From: ansuz Date: Fri, 23 Oct 2020 12:35:44 +0530 Subject: [PATCH 01/14] update changelog and version string for 3.23.2 (XerusDaamsi reloaded) --- CHANGELOG.md | 33 +++++++++++++++++++++++++++++++++ customize.dist/pages.js | 2 +- package-lock.json | 2 +- package.json | 2 +- 4 files changed, 36 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 30661d523..9f649883d 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,36 @@ +# XerusDaamsi reloaded (3.23.2) + +A number of instance administrators reported issues following our 3.23.1 release. We suspect the issues were caused by applying the recommended update steps out of order which would result in the incorrect HTTP header values getting cached for the most recent version of a file. Since the most recently updated headers modified some security settings, this caused a catastrophic error on clients receiving the incorrect headers which caused them to fail to load under certain circumstances. + +Regardless of the reasons behind this, we want CryptPad to be resilient against misconfiguration. This minor release includes a number of measures to override the unruly caching mechanisms employed internally by two of our most stubborn dependencies (CKEditor and OnlyOffice). Deploying 3.23.2 should force these editors to load the most recent versions of these dependencies according to the same policies as the rest of CryptPad and instruct clients to ignore any incorrect server responses they might have cached over the last few updates. + +This release also includes a number of bug fixes which had been tested in the meantime. + +Other bug fixes + +* We removed a hardcoded translation pertaining to the recently introduced "snapshot" functionality. +* Inspection of our server logs revealed a number of rare race conditions and type errors that have since been addressed. These included: + * multiple invocations of a callback when iterating over the list of all encrypted blobs + * a type error when recovering from the crash of one of the database worker processes + * premature closure of filesystem read-streams due to a timeout when the server was under heavy load +* A thorough review of our teams functionality revealed the possibility of some similarly rare issues that have since been corrected: + * it was possible to click the buttons on the "team invitation response dialog" multiple times before the first action completed. In some cases this could result in attempting to join a single team multiple times. + * it was also possible to activate trigger several actions that would modify your access rights for a team when the team had not fully synchronized with the server. Some of the time this was recoverable, but it could occasionally result in your team membership getting stuck in a bad state. + +We've implemented some measures to correct any team data that might have become corrupted due to the issues described above. Access rights from duplicated teams should be merged back into one set of cryptographic keys wherever possible. In cases where this isn't possible your role in the team will be automatically downgraded to the rank conferred by the keys you still have. For instance, somebody listed as an administrator who only has the keys required to view the team will downgrade themself to be a viewer. Subsequent promotions back to your previous team role should restore your possession of the required keys. + +To update to 3.23.2 from 3.23.0 or 3.23.1: + +Perform the same upgrade steps listed for 3.23.0 including the most recent configuration changes listed in `cryptpad/docs/example.nginx.conf... + +1. Modify your server's NGINX config file (but don't apply its changes until step 6) +2. Stop CryptPad's nodejs server +3. Get the latest platform code with git +4. Install client-side dependencies with `bower update` +5. Install server-side dependencies with `npm install` +6. Reload NGINX with `service nginx reload` to apply its config changes +7. Restart the CryptPad API server + # XerusDaamsi's revenge (3.23.1) We discovered a number of minor bugs after deploying 3.23.0. This minor release addresses them. diff --git a/customize.dist/pages.js b/customize.dist/pages.js index 2bd67b906..1db764bcc 100644 --- a/customize.dist/pages.js +++ b/customize.dist/pages.js @@ -62,7 +62,7 @@ define([ var imprintUrl = AppConfig.imprint && (typeof(AppConfig.imprint) === "boolean" ? '/imprint.html' : AppConfig.imprint); - Pages.versionString = "CryptPad v3.23.1 (XerusDaamsi's revenge)"; + Pages.versionString = "CryptPad v3.23.2 (XerusDaamsi reloaded)"; // used for the about menu Pages.imprintLink = AppConfig.imprint ? footLink(imprintUrl, 'imprint') : undefined; diff --git a/package-lock.json b/package-lock.json index 40d00411f..a9ee6e5e7 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,6 +1,6 @@ { "name": "cryptpad", - "version": "3.23.1", + "version": "3.23.2", "lockfileVersion": 1, "requires": true, "dependencies": { diff --git a/package.json b/package.json index 6f4fa275e..a849b2c2d 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "cryptpad", "description": "realtime collaborative visual editor with zero knowlege server", - "version": "3.23.1", + "version": "3.23.2", "license": "AGPL-3.0+", "repository": { "type": "git", From 69c26fe8c720101ed4090e592bab68916356e972 Mon Sep 17 00:00:00 2001 From: yflory Date: Fri, 23 Oct 2020 15:44:47 +0200 Subject: [PATCH 02/14] Fix various issues with restricted pads --- lib/historyKeeper.js | 4 ++ www/common/cryptget.js | 73 ++++++++++++++++++++++-- www/common/cryptpad-common.js | 93 ++++++++++++++++++++++++------- www/common/sframe-common-outer.js | 28 ++++++---- 4 files changed, 162 insertions(+), 36 deletions(-) diff --git a/lib/historyKeeper.js b/lib/historyKeeper.js index d95c0e99f..cfdb14717 100644 --- a/lib/historyKeeper.js +++ b/lib/historyKeeper.js @@ -80,6 +80,10 @@ module.exports.create = function (Env, cb) { return void cb(); } + // If the channel is restricted, send the history keeper ID so that they + // can try to authenticate + allowed.unshift(Env.id); + // otherwise they're not allowed. // respond with a special error that includes the list of keys // which would be allowed... diff --git a/www/common/cryptget.js b/www/common/cryptget.js index ab707a49b..e394788d7 100644 --- a/www/common/cryptget.js +++ b/www/common/cryptget.js @@ -1,12 +1,15 @@ define([ '/bower_components/chainpad-crypto/crypto.js', '/bower_components/chainpad-netflux/chainpad-netflux.js', + '/bower_components/netflux-websocket/netflux-client.js', '/common/common-util.js', '/common/common-hash.js', '/common/common-realtime.js', '/common/outer/network-config.js', + '/common/pinpad.js', + '/bower_components/nthen/index.js', '/bower_components/chainpad/chainpad.dist.js', -], function (Crypto, CPNetflux, Util, Hash, Realtime, NetConfig) { +], function (Crypto, CPNetflux, Netflux, Util, Hash, Realtime, NetConfig, Pinpad, nThen) { var finish = function (S, err, doc) { if (S.done) { return; } S.cb(err, doc); @@ -28,6 +31,50 @@ define([ } }; + var makeNetwork = function (cb) { + var wsUrl = NetConfig.getWebsocketURL(); + Netflux.connect(wsUrl).then(function (network) { + cb(null, network); + }, function (err) { + cb(err); + }); + }; + + var start = function (Session, config) { + // Create a network and authenticate with all our keys if necessary, + // then start chainpad-netflux + nThen(function (waitFor) { + if (Session.hasNetwork) { return; } + makeNetwork(waitFor(function (err, network) { + if (err) { return; } + config.network = network; + })); + }).nThen(function () { + Session.realtime = CPNetflux.start(config); + }); + }; + + var onRejected = function (config, Session, data, cb) { + // Check if we can authenticate + if (!Array.isArray(data) || !data.length || data[0].length !== 16) { + return void cb(true); + } + if (!Array.isArray(Session.accessKeys)) { return void cb(true); } + + // Authenticate + config.network.historyKeeper = data[0]; + nThen(function (waitFor) { + Session.accessKeys.forEach(function (obj) { + Pinpad.create(config.network, obj, waitFor(function (e) { + console.log('done', obj); + if (e) { console.error(e); } + })); + }); + }).nThen(function () { + cb(); + }); + }; + var makeConfig = function (hash, opt) { var secret; if (typeof(hash) === 'string') { @@ -67,7 +114,15 @@ define([ progress = progress || function () {}; var config = makeConfig(hash, opt); - var Session = { cb: cb, hasNetwork: Boolean(opt.network) }; + var Session = { + cb: cb, + accessKeys: opt.accessKeys, + hasNetwork: Boolean(opt.network) + }; + + config.onRejected = function (data, cb) { + onRejected(config, Session, data, cb); + }; config.onReady = function (info) { var rt = Session.session = info.realtime; @@ -95,7 +150,7 @@ define([ overwrite(config, opt); - Session.realtime = CPNetflux.start(config); + start(Session, config); }; var put = function (hash, doc, cb, opt) { @@ -105,7 +160,15 @@ define([ opt = opt || {}; var config = makeConfig(hash, opt); - var Session = { cb: cb, hasNetwork: Boolean(opt.network) }; + var Session = { + cb: cb, + accessKeys: opt.accessKeys, + hasNetwork: Boolean(opt.network) + }; + + config.onRejected = function (data, cb) { + onRejected(config, Session, data, cb); + }; config.onReady = function (info) { var realtime = Session.session = info.realtime; @@ -126,7 +189,7 @@ define([ }; overwrite(config, opt); - Session.session = CPNetflux.start(config); + start(Session, config); }; return { diff --git a/www/common/cryptpad-common.js b/www/common/cryptpad-common.js index 696627cbe..a9b2923a9 100644 --- a/www/common/cryptpad-common.js +++ b/www/common/cryptpad-common.js @@ -68,6 +68,38 @@ define([ }, cb); }; + common.getAccessKeys = function (cb) { + var keys = []; + Nthen(function (waitFor) { + // Push account keys + postMessage("GET", { + key: ['edPrivate'], + }, waitFor(function (obj) { + if (obj.error) { return; } + try { + keys.push({ + edPrivate: obj, + edPublic: Hash.getSignPublicFromPrivate(obj) + }); + } catch (e) { console.error(e); } + })); + // Push teams keys + postMessage("GET", { + key: ['teams'], + }, waitFor(function (obj) { + if (obj.error) { return; } + Object.keys(obj || {}).forEach(function (id) { + var t = obj[id]; + var _keys = t.keys.drive || {}; + if (!_keys.edPrivate) { return; } + keys.push(t.keys.drive); + }); + })); + }).nThen(function () { + cb(keys); + }); + }; + common.makeNetwork = function (cb) { require([ '/bower_components/netflux-websocket/netflux-client.js', @@ -629,6 +661,10 @@ define([ optsPut.password = password; })); } + common.getAccessKeys(waitFor(function (keys) { + optsGet.accessKeys = keys; + optsPut.accessKeys = keys; + })); }).nThen(function () { Crypt.get(parsed.hash, function (err, val) { if (err) { @@ -666,19 +702,28 @@ define([ password: data.password, initialState: parsed.type === 'poll' ? '{}' : undefined }; - Crypt.get(parsed.hash, _waitFor(function (err, _val) { - if (err) { - _waitFor.abort(); - return void cb(err); - } - try { - val = JSON.parse(_val); - fixPadMetadata(val, true); - } catch (e) { - _waitFor.abort(); - return void cb(e.message); - } - }), optsGet); + var next = _waitFor(); + Nthen(function (waitFor) { + // Authenticate in case the pad os restricted + common.getAccessKeys(waitFor(function (keys) { + optsGet.accessKeys = keys; + })); + }).nThen(function () { + Crypt.get(parsed.hash, function (err, _val) { + if (err) { + _waitFor.abort(); + return void cb(err); + } + try { + val = JSON.parse(_val); + fixPadMetadata(val, true); + next(); + } catch (e) { + _waitFor.abort(); + return void cb(e.message); + } + }, optsGet); + }); return; } @@ -741,9 +786,6 @@ define([ }).nThen(function () { Crypt.put(parsed2.hash, JSON.stringify(val), function () { cb(); - Crypt.get(parsed2.hash, function (err, val) { - console.warn(val); - }); }, optsPut); }); @@ -1006,7 +1048,7 @@ define([ oldSecret = Hash.getSecrets(parsed.type, parsed.hash, optsGet.password); oldChannel = oldSecret.channel; common.getPadMetadata({channel: oldChannel}, waitFor(function (metadata) { - oldMetadata = metadata; + oldMetadata = metadata || {}; })); common.getMetadata(waitFor(function (err, data) { if (err) { @@ -1058,6 +1100,11 @@ define([ if (expire) { optsPut.metadata.expire = (expire - (+new Date())) / 1000; // Lifetime in seconds } + }).nThen(function (waitFor) { + common.getAccessKeys(waitFor(function (keys) { + optsGet.accessKeys = keys; + optsPut.accessKeys = keys; + })); }).nThen(function (waitFor) { Crypt.get(parsed.hash, waitFor(function (err, val) { if (err) { @@ -1074,6 +1121,8 @@ define([ } }), optsGet); }).nThen(function (waitFor) { + optsPut.metadata.restricted = oldMetadata.restricted; + optsPut.metadata.allowed = oldMetadata.allowed; Crypt.put(newHash, cryptgetVal, waitFor(function (err) { if (err) { waitFor.abort(); @@ -1309,11 +1358,17 @@ define([ validateKey: newSecret.keys.validateKey }, }; + var optsGet = {}; Nthen(function (waitFor) { common.getPadAttribute('', waitFor(function (err, _data) { padData = _data; + optsGet.password = padData.password; }), href); + common.getAccessKeys(waitFor(function (keys) { + optsGet.accessKeys = keys; + optsPut.accessKeys = keys; + })); }).nThen(function (waitFor) { oldSecret = Hash.getSecrets(parsed.type, parsed.hash, padData.password); @@ -1392,9 +1447,7 @@ define([ waitFor.abort(); return void cb({ error: 'CANT_PARSE' }); } - }), { - password: padData.password - }); + }), optsGet); }).nThen(function (waitFor) { // Re-encrypt rtchannel oldRtChannel = Util.find(cryptgetVal, ['content', 'channel']); diff --git a/www/common/sframe-common-outer.js b/www/common/sframe-common-outer.js index bba393afb..aebafa834 100644 --- a/www/common/sframe-common-outer.js +++ b/www/common/sframe-common-outer.js @@ -1380,8 +1380,10 @@ define([ }; var i = 0; sframeChan.on('Q_CRYPTGET', function (data, cb) { + var keys; var todo = function () { data.opts.network = cgNetwork; + data.opts.accessKeys = keys; Cryptget.get(data.hash, function (err, val) { cb({ error: err, @@ -1400,17 +1402,21 @@ define([ cgNetwork = undefined; } i++; - if (!cgNetwork) { - cgNetwork = true; - return void Cryptpad.makeNetwork(function (err, nw) { - console.log(nw); - cgNetwork = nw; - todo(); - }); - } else if (cgNetwork === true) { - return void whenCGReady(todo); - } - todo(); + + Cryptpad.getAccessKeys(function (_keys) { + keys = _keys; + if (!cgNetwork) { + cgNetwork = true; + return void Cryptpad.makeNetwork(function (err, nw) { + console.log(nw); + cgNetwork = nw; + todo(); + }); + } else if (cgNetwork === true) { + return void whenCGReady(todo); + } + todo(); + }); }); sframeChan.on('EV_CRYPTGET_DISCONNECT', function () { if (!cgNetwork) { return; } From a8f53d04fc1825c06aabd9ca4d0c513ffe1b4b02 Mon Sep 17 00:00:00 2001 From: ansuz Date: Mon, 26 Oct 2020 17:24:35 +0530 Subject: [PATCH 03/14] proposed nginx configuration to enable XLSX export without disabling print from other apps --- docs/example.nginx.conf | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-) diff --git a/docs/example.nginx.conf b/docs/example.nginx.conf index 117eb2cc4..324df90c0 100644 --- a/docs/example.nginx.conf +++ b/docs/example.nginx.conf @@ -57,11 +57,6 @@ server { add_header Access-Control-Allow-Origin "*"; # add_header X-Frame-Options "SAMEORIGIN"; - # Enable SharedArrayBuffer in Firefox (for .xlsx export) - add_header Cross-Origin-Resource-Policy cross-origin; - add_header Cross-Origin-Opener-Policy same-origin; - add_header Cross-Origin-Embedder-Policy require-corp; - # Insert the path to your CryptPad repository root here root /home/cryptpad/cryptpad; index index.html; @@ -113,6 +108,14 @@ server { if ($uri = "/sheet/inner.html") { set $unsafe 1; } if ($uri ~ ^\/common\/onlyoffice\/.*\/index\.html.*$) { set $unsafe 1; } + set $coop ''; + if ($uri ~ ^\/sheet\/.*$) { set $coop 'same-origin'; } + + # Enable SharedArrayBuffer in Firefox (for .xlsx export) + add_header Cross-Origin-Resource-Policy cross-origin; + add_header Cross-Origin-Opener-Policy $coop; + add_header Cross-Origin-Embedder-Policy require-corp; + # everything except the sandbox domain is a privileged scope, as they might be used to handle keys if ($host != $sandbox_domain) { set $unsafe 0; } From f7bd3bdc23c57a6c9cb29b0c73cdc3ae0a9ba232 Mon Sep 17 00:00:00 2001 From: ansuz Date: Mon, 26 Oct 2020 17:34:34 +0530 Subject: [PATCH 04/14] don't pin falsey document ids --- lib/commands/pin-rpc.js | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/lib/commands/pin-rpc.js b/lib/commands/pin-rpc.js index 8b111438d..93d922620 100644 --- a/lib/commands/pin-rpc.js +++ b/lib/commands/pin-rpc.js @@ -162,7 +162,7 @@ Pinning.pinChannel = function (Env, safeKey, channels, cb) { // only pin channels which are not already pinned var toStore = channels.filter(function (channel) { - return pinned.indexOf(channel) === -1; + return channel && pinned.indexOf(channel) === -1; }); if (toStore.length === 0) { @@ -204,7 +204,7 @@ Pinning.unpinChannel = function (Env, safeKey, channels, cb) { // only unpin channels which are pinned var toStore = channels.filter(function (channel) { - return pinned.indexOf(channel) !== -1; + return channel && pinned.indexOf(channel) !== -1; }); if (toStore.length === 0) { From a1ee5943b46e714162a28163377783c6caf84c08 Mon Sep 17 00:00:00 2001 From: yflory Date: Mon, 26 Oct 2020 16:45:36 +0100 Subject: [PATCH 05/14] Improve tag UI --- .../src/less2/include/tokenfield.less | 16 ++++ www/common/common-interface.js | 76 ++++++++++++++++++- 2 files changed, 91 insertions(+), 1 deletion(-) diff --git a/customize.dist/src/less2/include/tokenfield.less b/customize.dist/src/less2/include/tokenfield.less index 42a06624c..e3d2c3e5d 100644 --- a/customize.dist/src/less2/include/tokenfield.less +++ b/customize.dist/src/less2/include/tokenfield.less @@ -20,6 +20,22 @@ margin: 0 10px; padding: 0; width: ~"calc(100% - 20px)"; + span.tokenfield-empty { + font-size: 14px; + font-style: italic; + color: lighten(@cryptpad_text_col, 10%); + } + .cp-tokenfield-container { + width: 100%; + } + .cp-tokenfield-form { + display: flex; + width: 100%; + input { + flex: 1; + min-width: 0 !important; + } + } .token { box-sizing: border-box; display: inline-flex; diff --git a/www/common/common-interface.js b/www/common/common-interface.js index 2e6c3a609..118931148 100644 --- a/www/common/common-interface.js +++ b/www/common/common-interface.js @@ -280,8 +280,82 @@ define([ }; var $root = $t.parent(); + + Messages.add = "Add"; // XXX + Messages.edit = "Edit"; // XXX + var $input = $root.find('.token-input'); + var $button = $(h('button.btn.btn-primary', [ + h('i.fa.fa-plus'), + h('span', Messages.add) + ])); + + + $button.click(function () { + $t.tokenfield('createToken', $input.val()); + }); + + var $container = $(h('span.cp-tokenfield-container')); + var $form = $(h('span.cp-tokenfield-form')); + $container.insertAfter($input); + + // Fix the UI to keep the "add" or "edit" button at the correct location + var isEdit = false; + var called = false; + var resetUI = function () { + called = true; + setTimeout(function () { + $container.find('.tokenfield-empty').remove(); + var $tokens = $root.find('.token').prependTo($container); + if (!$tokens.length) { + $container.prepend(h('span.tokenfield-empty', Messages.kanban_noTags)); + } + $form.append($input); + $form.append($button); + if (isEdit) { $button.find('span').text(Messages.edit); } + else { $button.find('span').text(Messages.add); } + $container.append($form); + $input.focus(); + isEdit = false; + called = false; + }); + }; + resetUI(); + + $t.on('tokenfield:removedtoken', function () { + resetUI(); + }); + $t.on('tokenfield:editedtoken', function () { + resetUI(); + }); + $t.on('tokenfield:createdtoken', function () { + $input.val(''); + resetUI(); + }); + $t.on('tokenfield:edittoken', function () { + isEdit = true; + }); + + // Fix UI issue where the input could go outside of the container + var MutationObserver = window.MutationObserver; + var observer = new MutationObserver(function(mutations) { + if (called) { return; } + mutations.forEach(function(mutation) { + for (var i = 0; i < mutation.addedNodes.length; i++) { + if (mutation.addedNodes[i].classList && + mutation.addedNodes[i].classList.contains('token-input')) { + resetUI(); + break; + } + } + }); + }); + observer.observe($root[0], { + childList: true, + subtree: false + }); + $t.on('tokenfield:removetoken', function () { - $root.find('.token-input').focus(); + $input.focus(); }); t.preventDuplicates = function (cb) { From a2b79d84b8c525a5b49e360e252b7543a85ad1cb Mon Sep 17 00:00:00 2001 From: ansuz Date: Tue, 27 Oct 2020 08:12:23 +0530 Subject: [PATCH 06/14] align nodejs http headers with example nginx --- docs/example.nginx.conf | 16 ++++++++-------- lib/defaults.js | 3 --- server.js | 16 ++++++++++++++-- 3 files changed, 22 insertions(+), 13 deletions(-) diff --git a/docs/example.nginx.conf b/docs/example.nginx.conf index 324df90c0..8319c657b 100644 --- a/docs/example.nginx.conf +++ b/docs/example.nginx.conf @@ -57,6 +57,14 @@ server { add_header Access-Control-Allow-Origin "*"; # add_header X-Frame-Options "SAMEORIGIN"; + set $coop ''; + if ($uri ~ ^\/sheet\/.*$) { set $coop 'same-origin'; } + + # Enable SharedArrayBuffer in Firefox (for .xlsx export) + add_header Cross-Origin-Resource-Policy cross-origin; + add_header Cross-Origin-Opener-Policy $coop; + add_header Cross-Origin-Embedder-Policy require-corp; + # Insert the path to your CryptPad repository root here root /home/cryptpad/cryptpad; index index.html; @@ -108,14 +116,6 @@ server { if ($uri = "/sheet/inner.html") { set $unsafe 1; } if ($uri ~ ^\/common\/onlyoffice\/.*\/index\.html.*$) { set $unsafe 1; } - set $coop ''; - if ($uri ~ ^\/sheet\/.*$) { set $coop 'same-origin'; } - - # Enable SharedArrayBuffer in Firefox (for .xlsx export) - add_header Cross-Origin-Resource-Policy cross-origin; - add_header Cross-Origin-Opener-Policy $coop; - add_header Cross-Origin-Embedder-Policy require-corp; - # everything except the sandbox domain is a privileged scope, as they might be used to handle keys if ($host != $sandbox_domain) { set $unsafe 0; } diff --git a/lib/defaults.js b/lib/defaults.js index 329e16f4c..4110e63d4 100644 --- a/lib/defaults.js +++ b/lib/defaults.js @@ -48,9 +48,6 @@ Default.httpHeaders = function () { "X-XSS-Protection": "1; mode=block", "X-Content-Type-Options": "nosniff", "Access-Control-Allow-Origin": "*", - "Cross-Origin-Resource-Policy": 'cross-origin', - "Cross-Origin-Opener-Policy": 'same-origin', - "Cross-Origin-Embedder-Policy": 'require-corp', }; }; diff --git a/server.js b/server.js index 0e0c2d79e..60247f47a 100644 --- a/server.js +++ b/server.js @@ -60,6 +60,10 @@ var app = Express(); } }()); +var applyHeaderMap = function (res, map) { + for (let header in map) { res.setHeader(header, map[header]); } +}; + var setHeaders = (function () { // load the default http headers unless the admin has provided their own via the config file var headers; @@ -96,14 +100,21 @@ var setHeaders = (function () { } if (Object.keys(headers).length) { return function (req, res) { + // apply a bunch of cross-origin headers for XLSX export in FF and printing elsewhere + applyHeaderMap(res, { + "Cross-Origin-Resource-Policy": 'cross-origin', + "Cross-Origin-Opener-Policy": /^\/sheet\//.test(req.url)? 'same-origin': '', + "Cross-Origin-Embedder-Policy": 'require-corp', + }); + + // targeted CSP, generic policies, maybe custom headers const h = [ - ///^\/pad\/inner\.html.*/, /^\/common\/onlyoffice\/.*\/index\.html.*/, /^\/(sheet|ooslide|oodoc)\/inner\.html.*/, ].some((regex) => { return regex.test(req.url); }) ? padHeaders : headers; - for (let header in h) { res.setHeader(header, h[header]); } + applyHeaderMap(res, h); }; } return function () {}; @@ -139,6 +150,7 @@ app.use(function (req, res, next) { setHeaders(req, res); if (/[\?\&]ver=[^\/]+$/.test(req.url)) { res.setHeader("Cache-Control", "max-age=31536000"); } + else { res.setHeader("Cache-Control", "no-cache"); } next(); }); From b46aaaed7b0a340c12f8710bd2deaeac6e264e9e Mon Sep 17 00:00:00 2001 From: ansuz Date: Tue, 27 Oct 2020 09:01:38 +0530 Subject: [PATCH 07/14] optimize Util.throttle to create fewer timeouts --- scripts/tests/throttle-test.js | 34 ++++++++++++++++++++++++++++++++ www/common/common-util.js | 36 ++++++++++++++++++++++++++++++++-- 2 files changed, 68 insertions(+), 2 deletions(-) create mode 100644 scripts/tests/throttle-test.js diff --git a/scripts/tests/throttle-test.js b/scripts/tests/throttle-test.js new file mode 100644 index 000000000..e84fc4a8a --- /dev/null +++ b/scripts/tests/throttle-test.js @@ -0,0 +1,34 @@ +var Util = require("../../lib/common-util"); + +(function (throttle) { + var last = 0; + var last_call = 0; + var f = Util.throttle(function (boop) { + var now = +new Date(); + if (last) { + console.log("last execution was %sms ago", now - last); + } else { + console.log("this is the first execution"); + } + last = now; + + //console.log('time of execution:', now); + console.log(boop); + }, 1000); + + [150, 250, 580, 850, 1500, 2200, 3990, 5000].forEach(function (delay) { + setTimeout(function () { + var now = +new Date(); + + if (last_call) { + console.log("last call was %sms ago", now - last_call); + } + + last_call = now; + //console.log("time of call for delay(%s):", delay, now); + f(delay); + }, delay); + }); +}(Util.throttle2)); + + diff --git a/www/common/common-util.js b/www/common/common-util.js index 0e86ecc8c..41797e7c8 100644 --- a/www/common/common-util.js +++ b/www/common/common-util.js @@ -306,11 +306,43 @@ }; Util.throttle = function (f, ms) { + var last = 0; var to; + var args; + + var defer = function (delay) { + // no timeout: run function `f` in `ms` milliseconds + // unless `g` is called again in the meantime + to = setTimeout(function () { + // wipe the current timeout handler + to = undefined; + + // take the current time + var now = +new Date(); + // compute time passed since `last` + var diff = now - last; + if (diff < ms) { + // don't run `f` if `g` was called since this timeout was set + // instead calculate how much further in the future your next + // timeout should be scheduled + return void defer(ms - diff); + } + + // else run `f` with the most recently supplied arguments + f.apply(null, args); + }, delay); + }; + var g = function () { - clearTimeout(to); - to = setTimeout(Util.bake(f, Util.slice(arguments)), ms); + // every time you call this function store the time + last = +new Date(); + // remember what arguments were passed + args = Util.slice(arguments); + // if there is a pending timeout then do nothing + if (to) { return; } + defer(ms); }; + g.clear = function () { clearTimeout(to); to = undefined; From c3fbbec72b04f8aa8fc057e0469c24fe6222f892 Mon Sep 17 00:00:00 2001 From: ansuz Date: Tue, 27 Oct 2020 12:46:27 +0530 Subject: [PATCH 08/14] lint compliance --- scripts/tests/throttle-test.js | 34 ---------------------------------- 1 file changed, 34 deletions(-) delete mode 100644 scripts/tests/throttle-test.js diff --git a/scripts/tests/throttle-test.js b/scripts/tests/throttle-test.js deleted file mode 100644 index e84fc4a8a..000000000 --- a/scripts/tests/throttle-test.js +++ /dev/null @@ -1,34 +0,0 @@ -var Util = require("../../lib/common-util"); - -(function (throttle) { - var last = 0; - var last_call = 0; - var f = Util.throttle(function (boop) { - var now = +new Date(); - if (last) { - console.log("last execution was %sms ago", now - last); - } else { - console.log("this is the first execution"); - } - last = now; - - //console.log('time of execution:', now); - console.log(boop); - }, 1000); - - [150, 250, 580, 850, 1500, 2200, 3990, 5000].forEach(function (delay) { - setTimeout(function () { - var now = +new Date(); - - if (last_call) { - console.log("last call was %sms ago", now - last_call); - } - - last_call = now; - //console.log("time of call for delay(%s):", delay, now); - f(delay); - }, delay); - }); -}(Util.throttle2)); - - From 034472d3ddd6a4b6c242f99f7c2826bdf3ec0df2 Mon Sep 17 00:00:00 2001 From: ansuz Date: Tue, 27 Oct 2020 12:47:15 +0530 Subject: [PATCH 09/14] move some implictly global state to env.js --- lib/commands/metadata.js | 4 +--- lib/env.js | 1 + 2 files changed, 2 insertions(+), 3 deletions(-) diff --git a/lib/commands/metadata.js b/lib/commands/metadata.js index 1d758564b..3d20f36e6 100644 --- a/lib/commands/metadata.js +++ b/lib/commands/metadata.js @@ -2,7 +2,6 @@ const Data = module.exports; const Meta = require("../metadata"); -const WriteQueue = require("../write-queue"); const Core = require("./core"); const Util = require("../common-util"); const HK = require("../hk-util"); @@ -53,7 +52,6 @@ Data.getMetadata = function (Env, channel, cb, Server, netfluxId) { value: value } */ -var queueMetadata = WriteQueue(); Data.setMetadata = function (Env, safeKey, data, cb, Server) { var unsafeKey = Util.unescapeKeyCharacters(safeKey); @@ -63,7 +61,7 @@ Data.setMetadata = function (Env, safeKey, data, cb, Server) { if (!command || typeof (command) !== 'string') { return void cb('INVALID_COMMAND'); } if (Meta.commands.indexOf(command) === -1) { return void cb('UNSUPPORTED_COMMAND'); } - queueMetadata(channel, function (next) { + Env.queueMetadata(channel, function (next) { Data.getMetadataRaw(Env, channel, function (err, metadata) { if (err) { cb(err); diff --git a/lib/env.js b/lib/env.js index 97ebd20ce..b1fc6680b 100644 --- a/lib/env.js +++ b/lib/env.js @@ -45,6 +45,7 @@ module.exports.create = function (config) { queueStorage: WriteQueue(), queueDeletes: WriteQueue(), queueValidation: WriteQueue(), + queueMetadata: WriteQueue(), batchIndexReads: BatchRead("HK_GET_INDEX"), batchMetadata: BatchRead('GET_METADATA'), From 1c986a81d1f2146016080ef0c572b4c0e5ec4f68 Mon Sep 17 00:00:00 2001 From: yflory Date: Tue, 27 Oct 2020 10:41:35 +0100 Subject: [PATCH 10/14] Fix Shared FOlder issues in the drive --- www/common/drive-ui.js | 3 +++ www/common/userObject.js | 3 ++- 2 files changed, 5 insertions(+), 1 deletion(-) diff --git a/www/common/drive-ui.js b/www/common/drive-ui.js index 649b5fe86..5d6a71b9d 100644 --- a/www/common/drive-ui.js +++ b/www/common/drive-ui.js @@ -4549,6 +4549,9 @@ define([ var rEl = manager.find(restorePath); if (manager.isFile(rEl)) { restoreName = manager.getTitle(rEl); + } else if (manager.isSharedFolder(rEl)) { + var sfData = manager.getSharedFolderData(rEl); + restoreName = sfData.title || sfData.lastTitle || Messages.fm_deletedFolder; } else { restoreName = restorePath[1]; } diff --git a/www/common/userObject.js b/www/common/userObject.js index 680ef3d0b..ca716f25f 100644 --- a/www/common/userObject.js +++ b/www/common/userObject.js @@ -846,7 +846,8 @@ define([ }; exp.ownedInTrash = function (isOwned) { return getFiles([TRASH]).map(function (id) { - var data = exp.getFileData(id); + var data = isSharedFolder(id) ? files[SHARED_FOLDERS][id] : exp.getFileData(id); + if (!data) { return; } return isOwned(data.owners) ? data.channel : undefined; }).filter(Boolean); }; From 63502abce834a1aeb6b0afb5b81bddd52aec1a7b Mon Sep 17 00:00:00 2001 From: yflory Date: Thu, 29 Oct 2020 14:39:23 +0100 Subject: [PATCH 11/14] Prompt to store edit link to your drive when you're a team viewer --- www/common/outer/async-store.js | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/www/common/outer/async-store.js b/www/common/outer/async-store.js index e849ce20d..290147427 100644 --- a/www/common/outer/async-store.js +++ b/www/common/outer/async-store.js @@ -1080,6 +1080,11 @@ define([ if (data.teamId && s.id !== data.teamId) { return; } if (storeLocally && s.id) { return; } + // If this is an edit link but we don't have edit rights, this entry is not useful + if (h.mode === "edit" && !s.secondaryKey) { + return; + } + var res = s.manager.findChannel(channel, true); if (res.length) { sendTo.push(s.id); From 83b318f156e57408617eb27e1f8e4c4139dc95aa Mon Sep 17 00:00:00 2001 From: yflory Date: Thu, 29 Oct 2020 14:44:14 +0100 Subject: [PATCH 12/14] Fix safe links with wrong access rights --- www/common/sframe-common-outer.js | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/www/common/sframe-common-outer.js b/www/common/sframe-common-outer.js index aebafa834..a03be3f64 100644 --- a/www/common/sframe-common-outer.js +++ b/www/common/sframe-common-outer.js @@ -317,10 +317,11 @@ define([ return void noPadData('NO_RESULT'); } // Data found but weaker? warn + expire = res.expire; if (edit && !res.href) { newHref = res.roHref; + return; } - expire = res.expire; // We have good data, keep the hash in memory newHref = edit ? res.href : (res.roHref || res.href); })); From fe30d5243cce101f52492e0da7ff738160df69f1 Mon Sep 17 00:00:00 2001 From: yflory Date: Thu, 29 Oct 2020 17:16:41 +0100 Subject: [PATCH 13/14] Fix 'store pad prompt' always displayed --- www/common/outer/async-store.js | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/www/common/outer/async-store.js b/www/common/outer/async-store.js index 290147427..d87bb955e 100644 --- a/www/common/outer/async-store.js +++ b/www/common/outer/async-store.js @@ -1081,7 +1081,7 @@ define([ if (storeLocally && s.id) { return; } // If this is an edit link but we don't have edit rights, this entry is not useful - if (h.mode === "edit" && !s.secondaryKey) { + if (h.mode === "edit" && s.id && !s.secondaryKey) { return; } From a3bb622df64ccea358debfe2c0ad1c76ec75004a Mon Sep 17 00:00:00 2001 From: yflory Date: Thu, 29 Oct 2020 17:26:23 +0100 Subject: [PATCH 14/14] Fix loading screen error --- customize.dist/loading.js | 2 ++ 1 file changed, 2 insertions(+) diff --git a/customize.dist/loading.js b/customize.dist/loading.js index 56e9a4cab..525f58e45 100644 --- a/customize.dist/loading.js +++ b/customize.dist/loading.js @@ -333,6 +333,8 @@ button.primary:hover{ window.CryptPad_loadingError = function (err) { if (!built) { return; } try { + var node = document.querySelector('.cp-loading-progress'); + if (node.parentNode) { node.parentNode.removeChild(node); } document.querySelector('.cp-loading-spinner-container').setAttribute('style', 'display:none;'); document.querySelector('#cp-loading-message').setAttribute('style', 'display:block;'); document.querySelector('#cp-loading-message').innerText = err;