diff --git a/CHANGELOG.md b/CHANGELOG.md index d33d0e25c..55d0e2534 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,3 +1,62 @@ +# 4.8.0 + +## Goals + +This release cycle we decided to give people a chance to try our forms app and provide feedback before we begin developing its second round of major features and improvements. In the meantime we planned to work mostly on the activities of our [NGI DAPSI](https://dapsi.ngi.eu/) project which concerns client-side file format conversions. Otherwise, we dedicated some of our independently funded time towards some internal code review and security best-practices as a follow-up to the recent quick-scan performed by [Radically Open Security](https://radicallyopensecurity.com/) that was funded by [NLnet](https://nlnet.nl) as a part of our now-closing _CryptPad for Communities_ project. + +## Update notes + +We are still accepting feedback concerning our Form application via [a form hosted on CryptPad.fr](https://cryptpad.fr/form/#/2/form/view/gYs4QS7DetInCXy0z2CQoUW6CwN6kaR2utGsftDzp58/). We will accept feedback here until July 12th, 2021, so if you'd like your opinions to be represented in the app's second round of development act quickly! + +Following our last release we sent out an email to the admins of each outdated instance that had included their addresses in the server's daily telemetry. This appears to have been successful, as more than half of the 700+ instances that provide this telemetry are now running **4.7.0**. Previously, only 15% of instances were running the latest version. It's worth noting that of those admins that are hosting the latest version, less than 10% have opted into future emails warning them of security issues. In case you missed it, this can be done on the admin panel's _Network_ tab. Unlike most companies, we consider excess data collection a liability rather than an asset. As such, adminstrator emails are no longer included in server telemetry unless the admin has consented to be contacted. + +The same HTTP request that communicates server telemetry will soon begin responding with the URL of our latest release notes if it is detected that the remote instance is running an older version. The admin panel's _Network_ tab for instances running 4.7.0 or later will begin prompting admins to view the release notes and update once 4.8.0 is available. + +The Network tab now includes a multiple choice form as well. If you have not disabled your instance's telemetry you can use this field to answer _why you run your instance_ (for a business, an academic institution, personal use, etc.). We intend to use this data to inform our development roadmap, though as always, the fastest way to get us to prioritize your needs is to contact us for a support contract (sales@cryptpad.fr). + +Server telemetry will also include an `installMethod` property. By default this is `"unspecified"`, but we are planning to work with packagers of alternate install methods to modify this property in their installation scripts. This will help us assess what proportion of instances are installed via the steps included in our installation guide vs other methods such as the various docker images. We hope that it will also allow us to determine the source of some common misconfigurations so we can propose some improvements to the root cause. + +Getting off the topic of telemetry: two types of data that were previously deleted outright (pin logs and login blocks) are now archived when the client sends a _remove_ command. This provides for the ability to restore old user credentials in cases where users claim that their new credentials do not work following a password change. Some discretion is required in such cases as a user might have intentionally invalidated their old credentials due to shoulder-surfing or the breach of another service's database where they'd reused credentials. Neither of these types of data are currently included in the scripts which evict old data as they are not likely to consume a significant amount of storage space. In any case, CryptPad's data is stored on the filesystem, so it's always possible to remove outdated files by removing them from `cryptpad/data/archive/*` or whatever path you've configured for your archives. + +This release introduces some minor changes to the provided NGINX configuration file to enable support for WebAssembly where it is required for client-side file format conversions. We've added some new tests on the /checkup/ page that determine whether these changes have been applied. This page can be found via a button on the admin panel. + +To update from 4.7.0 to 4.8.0: + +1. Apply the documented NGINX configuration +2. Stop your server +3. Get the latest code with git +4. Install the latest dependencies with `bower update` and `npm i` +5. Restart your server +6. Confirm that your instance is passing all the tests included on the `/checkup/` page + +## Features + +* Those who prefer using tools localized in Japanese can thank [@Suguru](https://mstdn.progressiv.dev/@suguru) for completing the Japanese translation of the platform's text! CryptPad is a fairly big platform with a lot of text to translate, so we really appreciate how much effort went into this. + * While we're on the topic, CryptPad's _Deutsch_ translation is kept up to date largely by a single member of the German Pirate Party (Piratenpartei Deutschland). This is a huge job and we appreciate your work too! + * Anyone else who wishes to give back to the project by doing the same can contribute translations on an ongoing basis through [our Weblate instance](https://weblate.cryptpad.fr/projects/cryptpad/app/). +* We've implemented a new app for file format conversions as a part of our _INTEROFFICE_ project. At this point this page is largely a test-case for the conversion engine that we hope to integrate more tightly into the rest of the platform. It allows users to load a variety of file formats into their browser and convert to any other format that has a defined conversion process from the original format. What's special about this is that files are converted entirely in your browser, unlike other platforms which do so in the cloud and expose their contents in the process. Currently we support conversion between the following formats in every browser that supports modern web standards (ie. not safari): + * XLSX and ODS + * DOCX and ODT and TXT + * PPTX and ODP +* In addition to the /convert/ page which supports office file formats, we also put some time into improving interoperability for our existing apps. We're introducing the ability to export rich text documents as Markdown (via the [turndown](https://github.com/mixmark-io/turndown) library), to import trello's JSON format into our Kanban app (with some loss of attributes because we don't support all the same features), and to export form summaries as CSV files. +* We've added another extension to our customized markdown renderer which replaces markdown images with a warning that CryptPad blocks remote content to prevent malicious users from tracking visitors to certain pages. Such images should already be blocked by our strict use of Content-Security-Policy headers, but this will provide a better indication why images are failing to load on isnstances that are correctly configured and a modest improvement to users' privacy on instances that aren't. +* Up until now it was possible to include style tags in markdown documents, which some of our more advanced users used in order to customize the appearance of their rendered documents. Unfortunately, these styles were not applied strictly to the markdown preview window, but to the page as a whole, making it possible to break the platform's interface (for that pad) through the use of overly broad and powerful style rules. As of this release style tags are now treated as special elements, such that their contents are compiled as [LESS](https://lesscss.org/) within a scope that is only applied to the preview pane. This was intended as a bug fix, but it's included here as a _feature_ because advanced users might see it as such and use it to do neat things. We have no funding for further work in this direction, however, and presently have no intent of providing documentation about this behaviour. +* The checkup page uses some slightly nicer methods of displaying values returned by tests when the expected value of `true` is not returned. Some tests have been revised to return the problematic value instead of `false` when the test fails, since there were some cases where it was not clear why the test was failing, such as when a header was present but duplicated. +* We've made some server requests related to _pinning files_ moderately faster by skipping an expensive calculation and omitting the value it returned. This value was meant to be used as a checksum to ensure that all of a user's documents were included in the list which should be associated with their account, however, clients used a separate command to fetch this checksum. The value provided in response to the other commands was never used by the client. +* We've implemented a system on the client for defining default templates for particular types of documents across an entire instance in addition to the use of documents in the _templates_ section of the users drive (or that of their teams). This is intended more as a generic system for us to reuse throughout the platform's source than an API for instance admins to use. If there is sufficient interest (and funding) from other admins we'll implement this as an instance configuration point. We now provide a _poll_ template to replicate the features of our old poll app which has been deprecated in favour of forms. +* We've included some more non-sensitive information about users' teams to the debugging data to which is automatically submitted along with support tickets, such as the id of the team's drive, roster, and how large the drive's contents are. +* The _Log out everywhere_ option that is displayed in the user admin menu in the top-right corner of the page for logged-in users now displays a confirmation before terminating all remote sessions. + +## Bug fixes + +* It was brought to our attention that the registration page was not trimming leading and trailing whitespace from usernames as intended. We've updated the page to do so, however, accounts created with such characters in their username field must enter their credentials exactly as they were at registration time in order to log in. We have no means of detecting such accounts on the server, as usernames are not visible to server admins. We'll consider this behaviour in the future if we introduce an option to change usernames as we do with passwords. +* We now double-check that login blocks (account credentials encrypted with a key derived from a username and password) can be accessed by the client when registering or changing passwords. It should be sufficient to rely on the server to report whether the encrypted credentials were stored successfully when uploading them, but in instances where these resources don't load due to a misbehaving browser extension it's better that we detect it at registration time rather than after the user creates content that will be difficult to access without assistance determining which extension or browser customization is to blame. +* We learned that the Javascript engine used on iOS has trouble parsing an alternative representation of data strings that every other platform seems to handle. This caused calendars to display incorrect data. Because Apple prevents third-party browsers from including their own JavaScript engines this means that users were affected by this Safari bug regardless of whether they used browsers branded as Safari, Firefox, Chrome, or otherwise. +* After some internal review we now guard against a variety of cases where user-crafted input could trigger a DOMException error and prevent a whole page worth of markdown content to fail to render. While there is no impact for users' privacy or security in this bug, a malicious user could exploit it to be annoying. +* Shortly after our last release a user reported being unable to access their account due to a typeError which we were able to [guard against](https://github.com/xwiki-labs/cryptpad/commit/abc9466abe71a76d1d31ef6a3c2c9bba4d2233e4). +* Images appearing in the 'lightbox' preview modal no longer appear stretched. +* Before applying actions that modify the team's membership we now confirm that server-enforced permissions match our local state. + # 4.7.0 ## Goals @@ -488,7 +547,7 @@ To upgrade from 3.24.0 to 3.25.0: ## Features * This release makes a lot of changes to how content is loaded over the network. - * Most notably, CryptPad now employs a client-side cache based on the the _indexedDB API_. Browsers that support this functionality will opportunistically store messages in a local cache for the next time they need them. This should make a considerable difference in how quickly you're able to load a pad, particularly if you accessing the server over a low-bandwidth network. + * Most notably, CryptPad now employs a client-side cache based on the _indexedDB API_. Browsers that support this functionality will opportunistically store messages in a local cache for the next time they need them. This should make a considerable difference in how quickly you're able to load a pad, particularly if you accessing the server over a low-bandwidth network. * Uploaded files (images, PDFs, etc.) are also cached in a similar way. Once you'd loaded an asset, your client will prefer to load its local copy instead of the server. * We've updated the code for our _full drive backup_ functionality so that it uses the local cache to load files more quickly. In addition to this, backing up the contents of your drive will also populate the cache as though you had loaded your documents in the normal fashion. This cache will persist until it is invalidated (due to the authoritative document having been deleted or had its history trimmed) or until you have logged out. * We've added the ability to configure the maximum size for automatically downloaded files. Any encrypted files that are above this size will instead require manual interaction to begin downloading. Files that are larger than this limit which are already loaded in your cache will still be automatically displayed. @@ -2008,7 +2067,7 @@ Finally, we prioritized the ability to archive files for a period instead of del * Users with existing friends on the platform will run a migration to allow them to share pads with friends directly instead of sending them a link. * they'll receive a notification indicating the title of the pad and who shared it * if you've already added friends on the platform, you can send them pads from the usual "sharing menu" -* Our code editor already offered the ability to set their color theme and highlighting mode, but now those values will be previewed when mousing over the the option in the dropdown. +* Our code editor already offered the ability to set their color theme and highlighting mode, but now those values will be previewed when mousing over the option in the dropdown. * Our slide editor now offers the same theme selection as the code editor * It's now possible to view the history of a shared folder by clicking the history button while viewing the shared folder's contents. diff --git a/config/config.example.js b/config/config.example.js index 69f0b1e91..96914fa92 100644 --- a/config/config.example.js +++ b/config/config.example.js @@ -276,4 +276,13 @@ module.exports = { * (false by default) */ verbose: false, + + /* Surplus information: + * + * 'installMethod' is included in server telemetry to voluntarily + * indicate how many instances are using unofficial installation methods + * such as Docker. + * + */ + installMethod: 'unspecified', }; diff --git a/customize.dist/login.js b/customize.dist/login.js index 74e5e86f6..3309190c0 100644 --- a/customize.dist/login.js +++ b/customize.dist/login.js @@ -153,7 +153,7 @@ define([ register: isRegister, }; - var RT, blockKeys, blockHash, Pinpad, rpc, userHash; + var RT, blockKeys, blockHash, blockUrl, Pinpad, rpc, userHash; nThen(function (waitFor) { // derive a predefined number of bytes from the user's inputs, @@ -171,7 +171,7 @@ define([ // the rest of their data // determine where a block for your set of keys would be stored - var blockUrl = Block.getBlockUrl(res.opt.blockKeys); + blockUrl = Block.getBlockUrl(res.opt.blockKeys); // Check whether there is a block at that location Util.fetch(blockUrl, waitFor(function (err, block) { @@ -412,12 +412,21 @@ define([ toPublish.edPublic = RT.proxy.edPublic; var blockRequest = Block.serialize(JSON.stringify(toPublish), res.opt.blockKeys); - rpc.writeLoginBlock(blockRequest, waitFor(function (e) { if (e) { console.error(e); + waitFor.abort(); return void cb(e); } + })); + }).nThen(function (waitFor) { + // confirm that the block was actually written before considering registration successful + Util.fetch(blockUrl, waitFor(function (err /*, block */) { + if (err) { + console.error(err); + waitFor.abort(); + return void cb(err); + } console.log("blockInfo available at:", blockHash); LocalStore.setBlockHash(blockHash); diff --git a/customize.dist/pages.js b/customize.dist/pages.js index b0fd2f20a..18632f6ea 100644 --- a/customize.dist/pages.js +++ b/customize.dist/pages.js @@ -105,7 +105,7 @@ define([ var imprintUrl = AppConfig.imprint && (typeof(AppConfig.imprint) === "boolean" ? '/imprint.html' : AppConfig.imprint); - Pages.versionString = "v4.7.0"; + Pages.versionString = "v4.8.0"; // used for the about menu diff --git a/customize.dist/src/less2/include/colortheme-dark.less b/customize.dist/src/less2/include/colortheme-dark.less index f8ea0c92b..5c1649850 100644 --- a/customize.dist/src/less2/include/colortheme-dark.less +++ b/customize.dist/src/less2/include/colortheme-dark.less @@ -53,7 +53,7 @@ @cryptpad_color_light_blue: #00b7d8; @cryptpad_color_red: #ff1100; @cryptpad_color_red_fade: fade(@cryptpad_color_red, 50%); -@cryptpad_color_red_fader: fade(@cryptpad_color_red, 25%); +@cryptpad_color_red_fader: fade(@cryptpad_color_red, 15%); @cryptpad_color_warn_red: @cryptpad_color_red_fade; @cryptpad_color_dark_red: #9e0000; @cryptpad_color_light_red: #FFD4D4; diff --git a/customize.dist/src/less2/include/colortheme.less b/customize.dist/src/less2/include/colortheme.less index a7b103797..18dcb1c0a 100644 --- a/customize.dist/src/less2/include/colortheme.less +++ b/customize.dist/src/less2/include/colortheme.less @@ -53,7 +53,7 @@ @cryptpad_color_light_blue: #00b7d8; @cryptpad_color_red: #ff1100; @cryptpad_color_red_fade: fade(@cryptpad_color_red, 50%); -@cryptpad_color_red_fader: fade(@cryptpad_color_red, 25%); +@cryptpad_color_red_fader: fade(@cryptpad_color_red, 15%); @cryptpad_color_warn_red: @cryptpad_color_red_fade; @cryptpad_color_dark_red: #9e0000; @cryptpad_color_light_red: #FFD4D4; diff --git a/customize.dist/src/less2/include/markdown.less b/customize.dist/src/less2/include/markdown.less index 3943f128d..43732187a 100644 --- a/customize.dist/src/less2/include/markdown.less +++ b/customize.dist/src/less2/include/markdown.less @@ -154,6 +154,38 @@ color: @cp_markdown-block-fg; text-align: left; } + + div.cp-inline-img-warning { + display: inline-block; + padding: 10px; + color: @cryptpad_text_col; + background-color: @cryptpad_color_red_fader; + border: 1px solid @cryptpad_color_red; + .cp-inline-img { + display: flex; + margin-bottom: 10px; + } + .cp-alt-txt { + margin-left: 10px; + font-family: monospace; + font-size: 0.8em; + color: fade(@cryptpad_text_col, 90%); + } + a { + color: @cryptpad_text_col; + font-size: 0.8em; + &.cp-remote-img::before { // .fa.fa-question-circle + font-family: FontAwesome; + //content: "\f08e\00a0"; + content: "\f08e\00a0\00a0"; + } + &.cp-learn-more::before { // .fa.fa-external-link + font-family: FontAwesome; + content: "\f059\00a0"; + //content: "\f059\00a0\00a0"; + } + } + } } .markdown_cryptpad() { diff --git a/customize.dist/src/less2/include/modals-ui-elements.less b/customize.dist/src/less2/include/modals-ui-elements.less index 92fef68d0..7ec19a699 100644 --- a/customize.dist/src/less2/include/modals-ui-elements.less +++ b/customize.dist/src/less2/include/modals-ui-elements.less @@ -221,6 +221,9 @@ button { line-height: 1.5; } + img { + align-self: center; + } & > iframe { width: 100%; height: 100%; diff --git a/docs/example.nginx.conf b/docs/example.nginx.conf index 85f42dd81..8bc47d9f8 100644 --- a/docs/example.nginx.conf +++ b/docs/example.nginx.conf @@ -64,7 +64,7 @@ server { add_header Permissions-Policy interest-cohort=(); set $coop ''; - if ($uri ~ ^\/(sheet|presentation|doc)\/.*$) { set $coop 'same-origin'; } + if ($uri ~ ^\/(sheet|presentation|doc|convert)\/.*$) { set $coop 'same-origin'; } # Enable SharedArrayBuffer in Firefox (for .xlsx export) add_header Cross-Origin-Resource-Policy cross-origin; @@ -214,7 +214,7 @@ server { # The nodejs server has some built-in forwarding rules to prevent # URLs like /pad from resulting in a 404. This simply adds a trailing slash # to a variety of applications. - location ~ ^/(register|login|settings|user|pad|drive|poll|slide|code|whiteboard|file|media|profile|contacts|todo|filepicker|debug|kanban|sheet|support|admin|notifications|teams|calendar|presentation|doc|form|report)$ { + location ~ ^/(register|login|settings|user|pad|drive|poll|slide|code|whiteboard|file|media|profile|contacts|todo|filepicker|debug|kanban|sheet|support|admin|notifications|teams|calendar|presentation|doc|form|report|convert)$ { rewrite ^(.*)$ $1/ redirect; } diff --git a/lib/commands/admin-rpc.js b/lib/commands/admin-rpc.js index ab48a1086..59750571e 100644 --- a/lib/commands/admin-rpc.js +++ b/lib/commands/admin-rpc.js @@ -326,6 +326,7 @@ var instanceStatus = function (Env, Server, cb) { blockDailyCheck: Env.blockDailyCheck, updateAvailable: Env.updateAvailable, + instancePurpose: Env.instancePurpose, }); }; diff --git a/lib/commands/block.js b/lib/commands/block.js index 8180cb68e..4af32af70 100644 --- a/lib/commands/block.js +++ b/lib/commands/block.js @@ -1,14 +1,10 @@ /*jshint esversion: 6 */ /* globals Buffer*/ -var Block = module.exports; - -const Fs = require("fs"); -const Fse = require("fs-extra"); -const Path = require("path"); +const Block = module.exports; const Nacl = require("tweetnacl/nacl-fast"); const nThen = require("nthen"); - const Util = require("../common-util"); +const BlockStore = require("../storage/block"); /* We assume that the server is secured against MitM attacks @@ -31,7 +27,9 @@ const Util = require("../common-util"); author of the block, since we assume that the block will have been encrypted with xsalsa20-poly1305 which is authenticated. */ -var validateLoginBlock = function (Env, publicKey, signature, block, cb) { // FIXME BLOCKS +Block.validateLoginBlock = function (Env, publicKey, signature, block, _cb) { + var cb = Util.once(Util.mkAsync(_cb)); + // convert the public key to a Uint8Array and validate it if (typeof(publicKey) !== 'string') { return void cb('E_INVALID_KEY'); } @@ -69,21 +67,7 @@ var validateLoginBlock = function (Env, publicKey, signature, block, cb) { // FI // call back with (err) if unsuccessful if (!verified) { return void cb("E_COULD_NOT_VERIFY"); } - return void cb(null, u8_block); -}; - -var createLoginBlockPath = function (Env, publicKey) { // FIXME BLOCKS - // prepare publicKey to be used as a file name - var safeKey = Util.escapeKeyCharacters(publicKey); - - // validate safeKey - if (typeof(safeKey) !== 'string') { - return; - } - - // derive the full path - // /home/cryptpad/cryptpad/block/fg/fg32kefksjdgjkewrjksdfksjdfsdfskdjfsfd - return Path.join(Env.paths.block, safeKey.slice(0, 2), safeKey); + return void cb(null, block); }; Block.validateAncestorProof = function (Env, proof, _cb) { @@ -103,31 +87,26 @@ Block.validateAncestorProof = function (Env, proof, _cb) { return void cb('E_INVALID_ANCESTOR_PROOF'); } // else fall through to next step - }).nThen(function (w) { - var path = createLoginBlockPath(Env, pub); - Fs.access(path, Fs.constants.F_OK, w(function (err) { - if (!err) { return; } - w.abort(); // else - return void cb("E_MISSING_ANCESTOR"); - })); }).nThen(function () { - cb(void 0, pub); + BlockStore.check(Env, pub, function (err) { + if (err) { return void cb('E_MISSING_ANCESTOR'); } + cb(void 0, pub); + }); }); } catch (err) { return void cb(err); } }; -Block.writeLoginBlock = function (Env, safeKey, msg, _cb) { // FIXME BLOCKS +Block.writeLoginBlock = function (Env, safeKey, msg, _cb) { var cb = Util.once(Util.mkAsync(_cb)); - //console.log(msg); var publicKey = msg[0]; var signature = msg[1]; var block = msg[2]; var registrationProof = msg[3]; var previousKey; - var validatedBlock, parsed, path; + var validatedBlock, path; nThen(function (w) { if (Util.escapeKeyCharacters(publicKey) !== safeKey) { w.abort(); @@ -156,44 +135,26 @@ Block.writeLoginBlock = function (Env, safeKey, msg, _cb) { // FIXME BLOCKS previousKey = provenKey; })); }).nThen(function (w) { - validateLoginBlock(Env, publicKey, signature, block, w(function (e, _validatedBlock) { + Env.validateLoginBlock(publicKey, signature, block, w(function (e, _validatedBlock) { if (e) { w.abort(); return void cb(e); } - if (!(_validatedBlock instanceof Uint8Array)) { + if (typeof(_validatedBlock) !== 'string') { w.abort(); - return void cb('E_INVALID_BLOCK'); + return void cb('E_INVALID_BLOCK_RETURNED'); } validatedBlock = _validatedBlock; - - // derive the filepath - path = createLoginBlockPath(Env, publicKey); - - // make sure the path is valid - if (typeof(path) !== 'string') { - return void cb('E_INVALID_BLOCK_PATH'); - } - - parsed = Path.parse(path); - if (!parsed || typeof(parsed.dir) !== 'string') { - w.abort(); - return void cb("E_INVALID_BLOCK_PATH_2"); - } - })); - }).nThen(function (w) { - // make sure the path to the file exists - Fse.mkdirp(parsed.dir, w(function (e) { - if (e) { - w.abort(); - cb(e); - } })); }).nThen(function () { - // actually write the block - Fs.writeFile(path, Buffer.from(validatedBlock), { encoding: "binary", }, function (err) { - if (err) { return void cb(err); } + var buffer; + try { + buffer = Buffer.from(Nacl.util.decodeBase64(validatedBlock)); + } catch (err) { + return void cb('E_BLOCK_DESERIALIZATION'); + } + BlockStore.write(Env, publicKey, buffer, function (err) { Env.Log.info('BLOCK_WRITE_BY_OWNER', { safeKey: safeKey, blockId: publicKey, @@ -201,11 +162,13 @@ Block.writeLoginBlock = function (Env, safeKey, msg, _cb) { // FIXME BLOCKS previousKey: previousKey, path: path, }); - cb(); + cb(err); }); }); }; +const DELETE_BLOCK = Nacl.util.encodeBase64(Nacl.util.decodeUTF8('DELETE_BLOCK')); + /* When users write a block, they upload the block, and provide a signature proving that they deserve to be able to write to @@ -216,10 +179,11 @@ Block.writeLoginBlock = function (Env, safeKey, msg, _cb) { // FIXME BLOCKS information, we can just sign some constant and use that as proof. */ -Block.removeLoginBlock = function (Env, safeKey, msg, cb) { // FIXME BLOCKS +Block.removeLoginBlock = function (Env, safeKey, msg, _cb) { + var cb = Util.once(Util.mkAsync(_cb)); + var publicKey = msg[0]; var signature = msg[1]; - var block = Nacl.util.decodeUTF8('DELETE_BLOCK'); // clients and the server will have to agree on this constant nThen(function (w) { if (Util.escapeKeyCharacters(publicKey) !== safeKey) { @@ -227,26 +191,14 @@ Block.removeLoginBlock = function (Env, safeKey, msg, cb) { // FIXME BLOCKS return void cb("INCORRECT_KEY"); } }).nThen(function () { - validateLoginBlock(Env, publicKey, signature, block, function (e /*::, validatedBlock */) { + Env.validateLoginBlock(publicKey, signature, DELETE_BLOCK, function (e) { if (e) { return void cb(e); } - // derive the filepath - var path = createLoginBlockPath(Env, publicKey); - - // make sure the path is valid - if (typeof(path) !== 'string') { - return void cb('E_INVALID_BLOCK_PATH'); - } - - // FIXME COLDSTORAGE - Fs.unlink(path, function (err) { - Env.Log.info('DELETION_BLOCK_BY_OWNER_RPC', { + BlockStore.archive(Env, publicKey, function (err) { + Env.Log.info('ARCHIVAL_BLOCK_BY_OWNER_RPC', { publicKey: publicKey, - path: path, status: err? String(err): 'SUCCESS', }); - - if (err) { return void cb(err); } - cb(); + cb(err); }); }); }); diff --git a/lib/commands/pin-rpc.js b/lib/commands/pin-rpc.js index 93d922620..89afc4ee8 100644 --- a/lib/commands/pin-rpc.js +++ b/lib/commands/pin-rpc.js @@ -8,7 +8,7 @@ const nThen = require("nthen"); //const escapeKeyCharacters = Util.escapeKeyCharacters; const unescapeKeyCharacters = Util.unescapeKeyCharacters; -var sumChannelSizes = function (sizes) { +var sumChannelSizes = function (sizes) { // FIXME this synchronous code could be done by a worker return Object.keys(sizes).map(function (id) { return sizes[id]; }) .filter(function (x) { // only allow positive numbers @@ -115,8 +115,8 @@ Pinning.getTotalSize = function (Env, safeKey, cb) { */ Pinning.removePins = function (Env, safeKey, cb) { // FIXME respect the queue - Env.pinStore.removeChannel(safeKey, function (err) { - Env.Log.info('DELETION_PIN_BY_OWNER_RPC', { + Env.pinStore.archiveChannel(safeKey, function (err) { + Env.Log.info('ARCHIVAL_PIN_BY_OWNER_RPC', { safeKey: safeKey, status: err? String(err): 'SUCCESS', }); @@ -145,7 +145,7 @@ var getFreeSpace = Pinning.getFreeSpace = function (Env, safeKey, cb) { }); }; -var getHash = Pinning.getHash = function (Env, safeKey, cb) { +Pinning.getHash = function (Env, safeKey, cb) { getChannelList(Env, safeKey, function (channels) { Env.hashChannelList(channels, cb); }); @@ -166,12 +166,12 @@ Pinning.pinChannel = function (Env, safeKey, channels, cb) { }); if (toStore.length === 0) { - return void getHash(Env, safeKey, cb); + return void cb(); } getMultipleFileSize(Env, toStore, function (e, sizes) { if (typeof(sizes) === 'undefined') { return void cb(e); } - var pinSize = sumChannelSizes(sizes); + var pinSize = sumChannelSizes(sizes); // FIXME don't do this in the main thread... getFreeSpace(Env, safeKey, function (e, free) { if (typeof(free) === 'undefined') { @@ -186,7 +186,7 @@ Pinning.pinChannel = function (Env, safeKey, channels, cb) { toStore.forEach(function (channel) { session.channels[channel] = true; }); - getHash(Env, safeKey, cb); + cb(); }); }); }); @@ -208,7 +208,7 @@ Pinning.unpinChannel = function (Env, safeKey, channels, cb) { }); if (toStore.length === 0) { - return void getHash(Env, safeKey, cb); + return void cb(); } Env.pinStore.message(safeKey, JSON.stringify(['UNPIN', toStore, +new Date()]), @@ -217,20 +217,19 @@ Pinning.unpinChannel = function (Env, safeKey, channels, cb) { toStore.forEach(function (channel) { delete session.channels[channel]; }); - getHash(Env, safeKey, cb); + cb(); }); }); }; -Pinning.resetUserPins = function (Env, safeKey, channelList, cb) { +Pinning.resetUserPins = function (Env, safeKey, channelList, _cb) { + var cb = Util.once(Util.mkAsync(_cb)); if (!Array.isArray(channelList)) { return void cb('INVALID_PIN_LIST'); } var session = Core.getSession(Env.Sessions, safeKey); + if (!channelList.length) { - return void getHash(Env, safeKey, function (e, hash) { - if (e) { return cb(e); } - cb(void 0, hash); - }); + return void cb(); } var pins = {}; @@ -270,9 +269,7 @@ Pinning.resetUserPins = function (Env, safeKey, channelList, cb) { // update in-memory cache IFF the reset was allowed. session.channels = pins; - getHash(Env, safeKey, function (e, hash) { - cb(e, hash); - }); + cb(); }); }); }); diff --git a/lib/decrees.js b/lib/decrees.js index 960ece1ef..5f599705e 100644 --- a/lib/decrees.js +++ b/lib/decrees.js @@ -173,6 +173,9 @@ commands.SET_SUPPORT_MAILBOX = makeGenericSetter('supportMailbox', function (arg return args_isString(args) && Core.isValidPublicKey(args[0]); }); +// CryptPad_AsyncStore.rpc.send('ADMIN', [ 'ADMIN_DECREE', ['SET_INSTANCE_PURPOSE', ["development"]]], console.log) +commands.SET_INSTANCE_PURPOSE = makeGenericSetter('instancePurpose', args_isString); + // Maintenance: Empty string or an object with a start and end time var isNumber = function (value) { return typeof(value) === "number" && !isNaN(value); diff --git a/lib/env.js b/lib/env.js index 6f1717c09..3d58f11f7 100644 --- a/lib/env.js +++ b/lib/env.js @@ -20,6 +20,7 @@ var canonicalizeOrigin = function (s) { module.exports.create = function (config) { const Env = { version: Package.version, + installMethod: config.installMethod || undefined, httpUnsafeOrigin: canonicalizeOrigin(config.httpUnsafeOrigin), httpSafeOrigin: canonicalizeOrigin(config.httpSafeOrigin), @@ -122,7 +123,7 @@ module.exports.create = function (config) { maxWorkers: config.maxWorkers, disableIntegratedTasks: config.disableIntegratedTasks || false, - disableIntegratedEviction: config.disableIntegratedEviction || false, + disableIntegratedEviction: typeof(config.disableIntegratedEviction) === 'undefined'? true: config.disableIntegratedEviction, // XXX false, lastEviction: +new Date(), evictionReport: {}, commandTimers: {}, diff --git a/lib/historyKeeper.js b/lib/historyKeeper.js index daa041b5f..44afe0ae0 100644 --- a/lib/historyKeeper.js +++ b/lib/historyKeeper.js @@ -123,6 +123,8 @@ module.exports.create = function (Env, cb) { Store.create({ filePath: pinPath, archivePath: Env.paths.archive, + // indicate that archives should be put in a 'pins' archvie folder + volumeId: 'pins', }, w(function (err, s) { if (err) { throw err; } Env.pinStore = s; diff --git a/lib/stats.js b/lib/stats.js index d1da0e202..1dbbb2ad9 100644 --- a/lib/stats.js +++ b/lib/stats.js @@ -4,6 +4,7 @@ const Stats = module.exports; Stats.instanceData = function (Env) { var data = { version: Env.version, + installMethod: Env.installMethod, domain: Env.myDomain, subdomain: Env.mySubdomain, @@ -11,8 +12,10 @@ Stats.instanceData = function (Env) { httpUnsafeOrigin: Env.httpUnsafeOrigin, httpSafeOrigin: Env.httpSafeOrigin, - adminEmail: Env.adminEmail, + adminEmail: Env.consentToContact? Env.adminEmail: undefined, consentToContact: Boolean(Env.consentToContact), + + instancePurpose: Env.instancePurpose === 'noanswer'? undefined: Env.instancePurpose, }; /* We reserve the right to choose not to include instances diff --git a/lib/storage/block.js b/lib/storage/block.js new file mode 100644 index 000000000..1078f6d2e --- /dev/null +++ b/lib/storage/block.js @@ -0,0 +1,88 @@ +/*jshint esversion: 6 */ +const Block = module.exports; +const Util = require("../common-util"); +const Path = require("path"); +const Fs = require("fs"); +const Fse = require("fs-extra"); +const nThen = require("nthen"); + +Block.mkPath = function (Env, publicKey) { + // prepare publicKey to be used as a file name + var safeKey = Util.escapeKeyCharacters(publicKey); + + // validate safeKey + if (typeof(safeKey) !== 'string') { return; } + + // derive the full path + // /home/cryptpad/cryptpad/block/fg/fg32kefksjdgjkewrjksdfksjdfsdfskdjfsfd + return Path.join(Env.paths.block, safeKey.slice(0, 2), safeKey); +}; + +Block.mkArchivePath = function (Env, publicKey) { + // prepare publicKey to be used as a file name + var safeKey = Util.escapeKeyCharacters(publicKey); + + // validate safeKey + if (typeof(safeKey) !== 'string') { + return; + } + + // derive the full path + // /home/cryptpad/cryptpad/block/fg/fg32kefksjdgjkewrjksdfksjdfsdfskdjfsfd + return Path.join(Env.paths.archive, 'block', safeKey.slice(0, 2), safeKey); +}; + +Block.archive = function (Env, publicKey, _cb) { + var cb = Util.once(Util.mkAsync(_cb)); + + // derive the filepath + var currentPath = Block.mkPath(Env, publicKey); + + // make sure the path is valid + if (typeof(currentPath) !== 'string') { + return void cb('E_INVALID_BLOCK_PATH'); + } + + var archivePath = Block.mkArchivePath(Env, publicKey); + // make sure the path is valid + if (typeof(archivePath) !== 'string') { + return void cb('E_INVALID_BLOCK_ARCHIVAL_PATH'); + } + + Fse.move(currentPath, archivePath, { + overwrite: true, + }, cb); +}; + +Block.check = function (Env, publicKey, _cb) { // 'check' because 'exists' implies boolean + var cb = Util.once(Util.mkAsync(_cb)); + var path = Block.mkPath(Env, publicKey); + Fs.access(path, Fs.constants.F_OK, cb); +}; + +Block.write = function (Env, publicKey, buffer, _cb) { + var cb = Util.once(Util.mkAsync(_cb)); + var path = Block.mkPath(Env, publicKey); + if (typeof(path) !== 'string') { return void cb('INVALID_PATH'); } + var parsed = Path.parse(path); + + nThen(function (w) { + Fse.mkdirp(parsed.dir, w(function (err) { + if (!err) { return; } + w.abort(); + cb(err); + })); + }).nThen(function (w) { + Block.archive(Env, publicKey, w(function (/* err */) { + /* + we proceed even if there are errors. + it might be ENOENT (there is no file to archive) + or EACCES (bad filesystem permissions for the existing archived block?) + or lots of other things, none of which justify preventing the write + */ + })); + }).nThen(function () { + Fs.writeFile(path, buffer, { encoding: 'binary' }, cb); + }); +}; + diff --git a/lib/storage/file.js b/lib/storage/file.js index 825f14066..30e6c7644 100644 --- a/lib/storage/file.js +++ b/lib/storage/file.js @@ -51,7 +51,7 @@ var mkPath = function (env, channelId) { }; var mkArchivePath = function (env, channelId) { - return Path.join(env.archiveRoot, 'datastore', channelId.slice(0, 2), channelId) + '.ndjson'; + return Path.join(env.archiveRoot, env.volumeId, channelId.slice(0, 2), channelId) + '.ndjson'; }; var mkMetadataPath = function (env, channelId) { @@ -59,7 +59,7 @@ var mkMetadataPath = function (env, channelId) { }; var mkArchiveMetadataPath = function (env, channelId) { - return Path.join(env.archiveRoot, 'datastore', channelId.slice(0, 2), channelId) + '.metadata.ndjson'; + return Path.join(env.archiveRoot, env.volumeId, channelId.slice(0, 2), channelId) + '.metadata.ndjson'; }; var mkTempPath = function (env, channelId) { @@ -1044,6 +1044,9 @@ module.exports.create = function (conf, _cb) { var env = { root: conf.filePath || './datastore', archiveRoot: conf.archivePath || './data/archive', + // supply a volumeId if you want a store to archive channels to and from + // to its own subpath within the archive directory + volumeId: conf.volumeId || 'datastore', channels: { }, batchGetChannel: BatchRead('store_batch_channel'), }; @@ -1076,7 +1079,7 @@ module.exports.create = function (conf, _cb) { } })); // make sure the cold storage directory exists - Fse.mkdirp(env.archiveRoot, PERMISSIVE, w(function (err) { + Fse.mkdirp(Path.join(env.archiveRoot, env.volumeId), PERMISSIVE, w(function (err) { if (err && err.code !== 'EEXIST') { w.abort(); return void cb(err); diff --git a/lib/workers/db-worker.js b/lib/workers/db-worker.js index 8585cc3f6..2371240ec 100644 --- a/lib/workers/db-worker.js +++ b/lib/workers/db-worker.js @@ -66,6 +66,9 @@ const init = function (config, _cb) { Store.create({ filePath: config.pinPath, archivePath: config.archivePath, + // important to initialize the pinstore with its own volume id + // otherwise archived pin logs will get mixed in with channels + volumeId: 'pins', }, w(function (err, _pinStore) { if (err) { w.abort(); @@ -694,6 +697,10 @@ COMMANDS.VALIDATE_ANCESTOR_PROOF = function (data, cb) { Block.validateAncestorProof(Env, data && data.proof, cb); }; +COMMANDS.VALIDATE_LOGIN_BLOCK = function (data, cb) { + Block.validateLoginBlock(Env, data.publicKey, data.signature, data.block, cb); +}; + process.on('message', function (data) { if (!data || !data.txid || !data.pid) { return void process.send({ diff --git a/lib/workers/index.js b/lib/workers/index.js index 85c66eeb5..c2bfb5740 100644 --- a/lib/workers/index.js +++ b/lib/workers/index.js @@ -451,6 +451,15 @@ Workers.initialize = function (Env, config, _cb) { }, cb); }; + Env.validateLoginBlock = function (publicKey, signature, block, cb) { + sendCommand({ + command: 'VALIDATE_LOGIN_BLOCK', + publicKey: publicKey, + signature: signature, + block: block, + }, cb); + }; + cb(void 0); }); }; diff --git a/package-lock.json b/package-lock.json index ab294ec92..4d7992f82 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,6 +1,6 @@ { "name": "cryptpad", - "version": "4.7.0", + "version": "4.8.0", "lockfileVersion": 1, "requires": true, "dependencies": { diff --git a/package.json b/package.json index f9dfc2c14..9e18f05f0 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "cryptpad", "description": "realtime collaborative visual editor with zero knowlege server", - "version": "4.7.0", + "version": "4.8.0", "license": "AGPL-3.0+", "repository": { "type": "git", diff --git a/scripts/evict-archived.js b/scripts/evict-archived.js index 7f90f9ff5..255246e99 100644 --- a/scripts/evict-archived.js +++ b/scripts/evict-archived.js @@ -56,6 +56,8 @@ var prepareEnv = function (Env, cb) { Store.create({ filePath: config.pinPath, + // archive pin logs to their own subpath + volumeId: 'pins', }, w(function (err, _) { if (err) { w.abort(); diff --git a/scripts/evict-inactive.js b/scripts/evict-inactive.js index bf7e1ca5b..2521e014f 100644 --- a/scripts/evict-inactive.js +++ b/scripts/evict-inactive.js @@ -56,6 +56,8 @@ var prepareEnv = function (Env, cb) { Store.create({ filePath: config.pinPath, + // archive pin logs to their own subpath + volumeId: 'pins', }, w(function (err, _) { if (err) { w.abort(); diff --git a/scripts/find-html-translations.js b/scripts/find-html-translations.js index dabdbac4b..fc1d4a18e 100644 --- a/scripts/find-html-translations.js +++ b/scripts/find-html-translations.js @@ -9,6 +9,7 @@ var simpleTags = [ // FIXME "", + '', '

', '

', @@ -70,7 +71,7 @@ processLang(EN, 'en', true); [ 'ar', - 'bn_BD', + //'bn_BD', 'ca', 'de', 'es', @@ -86,11 +87,15 @@ processLang(EN, 'en', true); 'ro', 'ru', 'sv', - 'te', + //'te', 'tr', 'zh', ].forEach(function (lang) { - var map = require("../www/common/translations/messages." + lang + ".json"); - if (!Object.keys(map).length) { return; } - processLang(map, lang); + try { + var map = require("../www/common/translations/messages." + lang + ".json"); + if (!Object.keys(map).length) { return; } + processLang(map, lang); + } catch (err) { + console.error(err); + } }); diff --git a/scripts/lint-translations.js b/scripts/lint-translations.js new file mode 100644 index 000000000..a38cd615a --- /dev/null +++ b/scripts/lint-translations.js @@ -0,0 +1,8 @@ +// TODO unify the following scripts + // unused-translations.js + // find-html-translations + +// more linting + // Search for 'Cryptpad' string (should be 'CryptPad') + // Search English for -ise\s + diff --git a/scripts/tests/test-rpc.js b/scripts/tests/test-rpc.js index 0fd0abc37..b92cf6287 100644 --- a/scripts/tests/test-rpc.js +++ b/scripts/tests/test-rpc.js @@ -122,14 +122,11 @@ var createUser = function (config, cb) { }); })); }).nThen(function (w) { - user.rpc.reset([], w(function (err, hash) { + user.rpc.reset([], w(function (err) { if (err) { w.abort(); user.shutdown(); - return console.log("RESET_ERR"); - } - if (!hash || hash !== EMPTY_ARRAY_HASH) { - throw new Error("EXPECTED EMPTY ARRAY HASH"); + return console.log("TEST_RESET_ERR"); } })); }).nThen(function (w) { @@ -214,17 +211,17 @@ var createUser = function (config, cb) { // TODO check your quota usage }).nThen(function (w) { - user.rpc.unpin([user.mailboxChannel], w(function (err, hash) { + user.rpc.unpin([user.mailboxChannel], w(function (err) { if (err) { w.abort(); return void cb(err); } + })); + }).nThen(function (w) { + user.rpc.getServerHash(w(function (err, hash) { + console.log(hash); - if (hash[0] !== EMPTY_ARRAY_HASH) { - //console.log('UNPIN_RESPONSE', hash); - throw new Error("UNPIN_DIDNT_WORK"); - } - user.latestPinHash = hash[0]; + user.latestPinHash = hash; })); }).nThen(function (w) { // clean up the pin list to avoid lots of accounts on the server @@ -304,7 +301,8 @@ nThen(function (w) { }, w(function (err, roster) { if (err) { w.abort(); - return void console.trace(err); + console.error(err); + return void console.error("ROSTER_ERROR"); } oscar.roster = roster; oscar.destroy.reg(function () { diff --git a/server.js b/server.js index 848167a98..3cea171c6 100644 --- a/server.js +++ b/server.js @@ -90,7 +90,7 @@ var setHeaders = (function () { return function (req, res) { // apply a bunch of cross-origin headers for XLSX export in FF and printing elsewhere applyHeaderMap(res, { - "Cross-Origin-Opener-Policy": /^\/sheet\//.test(req.url)? 'same-origin': '', + "Cross-Origin-Opener-Policy": /^\/(sheet|presentation|doc|convert)\//.test(req.url)? 'same-origin': '', }); if (Env.NO_SANDBOX) { // handles correct configuration for local development diff --git a/www/admin/app-admin.less b/www/admin/app-admin.less index 17450361c..227792540 100644 --- a/www/admin/app-admin.less +++ b/www/admin/app-admin.less @@ -203,6 +203,17 @@ } } + .cp-admin-radio-container { + display: flex; + align-items: left; //center; + flex-wrap: wrap; + flex-direction: column; + label { + margin-right: 40px; + margin-top: 5px; + } + } + .cp-admin-broadcast-form { input.flatpickr-input { width: 307.875px !important; // same width as flatpickr calendar diff --git a/www/admin/inner.js b/www/admin/inner.js index f8bcad681..263f9a9bc 100644 --- a/www/admin/inner.js +++ b/www/admin/inner.js @@ -94,6 +94,7 @@ define([ 'cp-admin-list-my-instance', 'cp-admin-consent-to-contact', 'cp-admin-remove-donate-button', + 'cp-admin-instance-purpose', ], }; @@ -1853,6 +1854,73 @@ define([ }, }); + var sendDecree = function (data, cb) { + sFrameChan.query('Q_ADMIN_RPC', { + cmd: 'ADMIN_DECREE', + data: data, + }, cb); + }; + + create['instance-purpose'] = function () { + var key = 'instance-purpose'; + var $div = makeBlock(key); // Messages.admin_instancePurposeTitle.admin_instancePurposeHint + + var values = [ + 'noanswer', // Messages.admin_purpose_noanswer + 'experiment', // Messages.admin_purpose_experiment + 'personal', // Messages.admin_purpose_personal + 'education', // Messages.admin_purpose_education + 'org', // Messages.admin_purpose_org + 'business', // Messages.admin_purpose_business + 'public', // Messages.admin_purpose_public + ]; + + var defaultPurpose = 'noanswer'; + var purpose = APP.instanceStatus.instancePurpose || defaultPurpose; + + var opts = h('div.cp-admin-radio-container', [ + values.map(function (key) { + var full_key = 'admin_purpose_' + key; + return UI.createRadio('cp-instance-purpose-radio', 'cp-instance-purpose-radio-'+key, + Messages[full_key] || Messages._getKey(full_key, [defaultPurpose]), + key === purpose, { + input: { value: key }, + label: { class: 'noTitle' } + }); + }) + ]); + + var $opts = $(opts); + //var $br = $(h('br',)); + //$div.append($br); + + $div.append(opts); + + var setPurpose = function (value, cb) { + sendDecree([ + 'SET_INSTANCE_PURPOSE', + [ value] + ], cb); + }; + + $opts.on('change', function () { + var val = $opts.find('input:radio:checked').val(); + console.log(val); + //spinner.spin(); + setPurpose(val, function (e, response) { + if (e || response.error) { + UI.warn(Messages.error); + //spinner.hide(); + return; + } + //spinner.done(); + UI.log(Messages.saved); + }); + }); + + return $div; + }; + var hideCategories = function () { APP.$rightside.find('> div').hide(); }; diff --git a/www/calendar/inner.js b/www/calendar/inner.js index 58ad22ac1..21cb12789 100644 --- a/www/calendar/inner.js +++ b/www/calendar/inner.js @@ -17,6 +17,7 @@ define([ '/customize/application_config.js', '/lib/calendar/tui-calendar.min.js', '/calendar/export.js', + '/lib/datepicker/flatpickr.js', '/common/inner/share.js', '/common/inner/access.js', @@ -46,6 +47,7 @@ define([ AppConfig, Calendar, Export, + Flatpickr, Share, Access, Properties ) { @@ -169,9 +171,9 @@ define([ var obj = data.content[uid]; obj.title = obj.title || ""; obj.location = obj.location || ""; - if (obj.isAllDay && obj.startDay) { obj.start = +new Date(obj.startDay); } + if (obj.isAllDay && obj.startDay) { obj.start = +Flatpickr.parseDate((obj.startDay)); } if (obj.isAllDay && obj.endDay) { - var endDate = new Date(obj.endDay); + var endDate = Flatpickr.parseDate(obj.endDay); endDate.setHours(23); endDate.setMinutes(59); endDate.setSeconds(59); diff --git a/www/checkup/main.js b/www/checkup/main.js index 925cd4b7b..f4a40d11e 100644 --- a/www/checkup/main.js +++ b/www/checkup/main.js @@ -732,6 +732,84 @@ define([ cb(isHTTPS(trimmedUnsafe) && isHTTPS(trimmedSafe)); }); + [ + 'sheet', + 'presentation', + 'doc', + 'convert', + ].forEach(function (url) { + assert(function (cb, msg) { + var header = 'cross-origin-opener-policy'; + var expected = 'same-origin'; + deferredPostMessage({ + command: 'GET_HEADER', + content: { + url: '/' + url + '/', + header: header, + } + }, function (content) { + msg.appendChild(h('span', [ + code(url), + ' was served without the correct ', + code(header), + ' HTTP header value (', + code(expected), + '). This will interfere with your ability to convert between office file formats.' + ])); + cb(content === expected); + }); + }); + }); + +/* + assert(function (cb, msg) { + setWarningClass(msg); + $.ajax(cacheBuster('/'), { + dataType: 'text', + complete: function (xhr) { + var serverToken = xhr.getResponseHeader('server'); + if (serverToken === null) { return void cb(true); } + + var lowered = (serverToken || '').toLowerCase(); + var family; + + ['Apache', 'Caddy', 'NGINX'].some(function (pattern) { + if (lowered.indexOf(pattern.toLowerCase()) !== -1) { + family = pattern; + return true; + } + }); + + var text = [ + "This instance is set to respond with an HTTP ", + code("server"), + " header. This information can make it easier for attackers to find and exploit known vulnerabilities. ", + ]; + + if (family === 'NGINX') { // FIXME incorrect instructions for HTTP2. needs a recompile? + msg.appendChild(h('span', text.concat([ + "This can be addressed by setting ", + code("server_tokens off"), + " in your global NGINX config." + ]))); + return void cb(serverToken); + } + + // handle other + msg.appendChild(h('span', text.concat([ + "In this case, it appears that the host server is running ", + code(serverToken), + " instead of ", + code("NGINX"), + " as recommended. As such, you may not benefit from the latest security enhancements that are tested and maintained by the CryptPad development team.", + ]))); + + cb(serverToken); + } + }); + }); +*/ + if (false) { assert(function (cb, msg) { msg.innerText = 'fake test to simulate failure'; diff --git a/www/checkup/sandbox/main.js b/www/checkup/sandbox/main.js index 7ddfb07f8..e11aa1d52 100644 --- a/www/checkup/sandbox/main.js +++ b/www/checkup/sandbox/main.js @@ -27,12 +27,14 @@ define([ }; window.addEventListener("message", function (event) { + var txid, command; if (event && event.data) { try { //console.log(JSON.parse(event.data)); var msg = JSON.parse(event.data); - var command = msg.command; - var txid = msg.txid; + command = msg.command; + txid = msg.txid; + if (!txid) { return; } COMMANDS[command](msg.content, function (response) { // postMessage with same txid postMessage({ @@ -41,7 +43,11 @@ define([ }); }); } catch (err) { - console.error(err); + postMessage({ + txid: txid, + content: err, + }); + console.error(err, command); } } else { console.error(event); diff --git a/www/code/inner.js b/www/code/inner.js index d74513035..785e82788 100644 --- a/www/code/inner.js +++ b/www/code/inner.js @@ -507,7 +507,7 @@ define([ var fileHost = privateData.fileHost || privateData.origin; var src = fileHost + Hash.getBlobPathFromHex(secret.channel); var key = Hash.encodeBase64(secret.keys.cryptKey); - var mt = ''; + var mt = UI.mediaTag(src, key).outerHTML; editor.replaceSelection(mt); } }; diff --git a/www/common/application_config_internal.js b/www/common/application_config_internal.js index ad5f726f8..15e5b5e34 100644 --- a/www/common/application_config_internal.js +++ b/www/common/application_config_internal.js @@ -12,7 +12,7 @@ define(function() { * You should never remove the drive from this list. */ AppConfig.availablePadTypes = ['drive', 'teams', 'pad', 'sheet', 'code', 'slide', 'poll', 'kanban', 'whiteboard', - /*'doc', 'presentation',*/ 'file', /*'todo',*/ 'contacts', 'form']; + /*'doc', 'presentation',*/ 'file', /*'todo',*/ 'contacts', 'form', 'convert']; /* The registered only types are apps restricted to registered users. * You should never remove apps from this list unless you know what you're doing. The apps * listed here by default can't work without a user account. @@ -22,6 +22,12 @@ define(function() { */ AppConfig.registeredOnlyTypes = ['file', 'contacts', 'notifications', 'support']; + // to prevent apps that aren't officially supported from showing up + // in the document creation modal + AppConfig.hiddenTypes = ['drive', 'teams', 'contacts', 'todo', 'file', 'accounts', 'calendar', 'poll', 'convert', + //'doc', 'presentation' + ]; + /* CryptPad is available is multiple languages, but only English and French are maintained * by the developers. The other languages may be outdated, and any missing string for a langauge * will use the english version instead. You can customize the langauges you want to be available diff --git a/www/common/common-interface.js b/www/common/common-interface.js index 9bdf9ec7a..4d3520426 100644 --- a/www/common/common-interface.js +++ b/www/common/common-interface.js @@ -41,6 +41,15 @@ define([ return e; }; + // FIXME almost everywhere this is used would also be + // a good candidate for sframe-common's getMediatagFromHref + UI.mediaTag = function (src, key) { + return h('media-tag', { + src: src, + 'data-crypto-key': 'cryptpad:' + key, + }); + }; + var findCancelButton = UI.findCancelButton = function (root) { if (root) { return $(root).find('button.cancel').last(); diff --git a/www/common/common-ui-elements.js b/www/common/common-ui-elements.js index 3e27b4ee7..0ab010cb5 100644 --- a/www/common/common-ui-elements.js +++ b/www/common/common-ui-elements.js @@ -1886,8 +1886,11 @@ define([ }, content: h('span', Messages.logoutEverywhere), action: function () { - Common.getSframeChannel().query('Q_LOGOUT_EVERYWHERE', null, function () { - Common.gotoURL(origin + '/'); + UI.confirm(Messages.settings_logoutEverywhereConfirm, function (yes) { + if (!yes) { return; } + Common.getSframeChannel().query('Q_LOGOUT_EVERYWHERE', null, function () { + Common.gotoURL(origin + '/'); + }); }); }, }); @@ -2075,15 +2078,9 @@ define([ var $container = $('
'); var i = 0; + var types = AppConfig.availablePadTypes.filter(function (p) { - if (p === 'drive') { return; } - if (p === 'teams') { return; } - if (p === 'contacts') { return; } - if (p === 'todo') { return; } - if (p === 'file') { return; } - if (p === 'accounts') { return; } - if (p === 'calendar') { return; } - if (p === 'poll') { return; } // Replaced by forms + if (AppConfig.hiddenTypes.indexOf(p) !== -1) { return; } if (!common.isLoggedIn() && AppConfig.registeredOnlyTypes && AppConfig.registeredOnlyTypes.indexOf(p) !== -1) { return; } return true; diff --git a/www/common/cryptpad-common.js b/www/common/cryptpad-common.js index 9eb5398e7..15234c35e 100644 --- a/www/common/cryptpad-common.js +++ b/www/common/cryptpad-common.js @@ -1937,6 +1937,17 @@ define([ waitFor.abort(); return void cb(obj); } + })); + }).nThen(function (waitFor) { + var blockUrl = Block.getBlockUrl(blockKeys); + Util.fetch(blockUrl, waitFor(function (err /* block */) { + if (err) { + console.error(err); + waitFor.abort(); + return cb({ + error: err, + }); + } console.log("new login block written"); var newBlockHash = Block.getBlockHash(blockKeys); LocalStore.setBlockHash(newBlockHash); diff --git a/www/common/diffMarked.js b/www/common/diffMarked.js index cb95f37aa..0fe8aed0a 100644 --- a/www/common/diffMarked.js +++ b/www/common/diffMarked.js @@ -8,11 +8,14 @@ define([ '/common/inner/common-mediatag.js', '/common/media-tag.js', '/customize/messages.js', + '/common/less.min.js', + '/customize/pages.js', + '/common/highlight/highlight.pack.js', '/lib/diff-dom/diffDOM.js', '/bower_components/tweetnacl/nacl-fast.min.js', 'css!/common/highlight/styles/'+ (window.CryptPad_theme === 'dark' ? 'dark.css' : 'github.css') -],function ($, ApiConfig, Marked, Hash, Util, h, MT, MediaTag, Messages) { +],function ($, ApiConfig, Marked, Hash, Util, h, MT, MediaTag, Messages, Less, Pages) { var DiffMd = {}; var Highlight = window.hljs; @@ -172,12 +175,16 @@ define([ return h('div.cp-md-toc', content).outerHTML; }; - DiffMd.render = function (md, sanitize, restrictedMd) { + var noHeadingId = false; + DiffMd.render = function (md, sanitize, restrictedMd, noId) { Marked.setOptions({ renderer: restrictedMd ? restrictedRenderer : renderer, }); + noHeadingId = noId; var r = Marked(md, { - sanitize: sanitize + sanitize: sanitize, + headerIds: !noId, + gfm: true, }); // Add Table of Content @@ -207,7 +214,11 @@ define([ }; restrictedRenderer.code = renderer.code; + var _heading = renderer.heading; renderer.heading = function (text, level) { + if (noHeadingId) { + return _heading.apply(this, arguments); + } var i = 0; var safeText = text.toLowerCase().replace(/[^\w]+/g, '-'); var getId = function () { @@ -266,9 +277,43 @@ define([ return '
  • ' + text + '
  • \n'; }; + + var qualifiedHref = function (href) { + if (typeof(window.URL) === 'undefined') { return href; } + try { + var url = new URL(href, ApiConfig.httpUnsafeOrigin); + return url.href; + } catch (err) { + console.error(err); + return href; + } + }; + + var isLocalURL = function (href) { + // treat all URLs as remote if you are using an ancient browser + if (typeof(window.URL) === 'undefined') { return false; } + try { + var url = new URL(href, ApiConfig.httpUnsafeOrigin); + // FIXME data URLs can be quite large, but that should be addressed + // in the source markdown's, not the renderer + if (url.protocol === 'data:') { return true; } + var localURL = new URL(ApiConfig.httpUnsafeOrigin); + return url.host === localURL.host; + } catch (err) { + return true; + } + }; + renderer.image = function (href, title, text) { + if (isLocalURL(href) && href.slice(0, 6) !== '/file/') { + return h('img', { + src: href, + title: title || '', + alt: text, + }).outerHTML; + } + if (href.slice(0,6) === '/file/') { - // DEPRECATED // Mediatag using markdown syntax should not be used anymore so they don't support // password-protected files console.log('DEPRECATED: mediatag using markdown syntax!'); @@ -276,19 +321,38 @@ define([ var secret = Hash.getSecrets('file', parsed.hash); var src = (ApiConfig.fileHost || '') +Hash.getBlobPathFromHex(secret.channel); var key = Hash.encodeBase64(secret.keys.cryptKey); - var mt = ''; - if (mediaMap[src]) { - mt += mediaMap[src]; - } - mt += ''; - return mt; - } - var out = '' + text + '' : '>'; - return out; + + var warning = h('div.cp-inline-img-warning', [ + h('div.cp-inline-img', [ + h('img.cp-inline-img', { + src: '/images/broken.png', + //title: title || '', // FIXME sort out tippy issues (double-title) + }), + h('p.cp-alt-txt', text), + ]), + h('span.cp-img-block-notice', { + }, Messages.resources_imageBlocked), + h('br'), + h('a.cp-remote-img', { + href: qualifiedHref(href), + }, [ + Messages.resources_openInNewTab + ]), + h('br'), + h('a.cp-learn-more', { + href: Pages.localizeDocsLink('https://docs.cryptpad.fr/en/user_guide/security.html#remote-content'), + }, [ + Messages.resources_learnWhy + ]), + ]); + + return warning.outerHTML; }; restrictedRenderer.image = renderer.image; @@ -473,6 +537,75 @@ define([ } }; + var applyCSS = function (el, css) { + var style = h('style'); + style.appendChild(document.createTextNode(css)); + el.innerText = ''; + el.appendChild(style); + }; + + // trim non-functional text from less input so that + // the compiler is only triggered when there has been a functional change + var canonicalizeLess = function (source) { + return (source || '') + // leading and trailing spaces are irrelevant + .trim() + // line comments are easy to disregard + .replace(/\/\/[^\n]*/g, '') + // lines with nothing but spaces and tabs can be ignored + .replace(/^[ \t]*$/g, '') + // consecutive newlines make no difference + .replace(/\n+/g, ''); + }; + + var rendered_less = {}; + var getRenderedLess = (function () { + var timeouts = {}; + return function (src) { + if (!rendered_less[src]) { return; } + if (timeouts[src]) { + clearTimeout(timeouts[src]); + } + // avoid memory leaks by deleting cached content + // 15s after it was last accessed + timeouts[src] = setTimeout(function () { + delete rendered_less[src]; + delete timeouts[src]; + }, 15000); + return rendered_less[src]; + }; + }()); + + plugins.less = { + name: 'less', + attr: 'less-src', + render: function renderLess ($el, opt) { + var src = canonicalizeLess($el.text()); + if (!src) { return; } + var el = $el[0]; + var rendered = getRenderedLess(src); + if (rendered) { return void applyCSS(el, rendered); } + + var scope = opt.scope.attr('id') || 'cp-app-code-preview-content'; + var scoped_src = '#' + scope + ' { ' + src + '}'; + //console.error("RENDERING LESS"); + Less.render(scoped_src, {}, function (err, result) { + // the console is the only feedback for users to know that they did something wrong + // but less rendering isn't intended so much as a feature but a useful tool to avoid + // leaking styles from the preview into the rest of the DOM. This is an improvement. + if (err) { + // we assume the compiler is deterministic. Something that returns an error once + // will do it again, so avoid successive calls by caching a truthy + // but non-functional string to block them. + rendered_less[src] = ' '; + return void console.error(err); + } + var css = rendered_less[src] = result.css; + applyCSS(el, css); + }); + }, + }; + var getAvailableCachedElement = function ($content, cache, src) { var cached = cache[src]; if (!Array.isArray(cached)) { return; } @@ -546,7 +679,8 @@ define([ // caching their source as you go $(newDomFixed).find('pre[data-plugin]').each(function (index, el) { if (el.childNodes.length === 1 && el.childNodes[0].nodeType === 3) { - var plugin = plugins[el.getAttribute('data-plugin')]; + var type = el.getAttribute('data-plugin'); + var plugin = plugins[type]; if (!plugin) { return; } var src = canonicalizeMermaidSource(el.childNodes[0].wholeText); el.setAttribute(plugin.attr, src); @@ -559,7 +693,8 @@ define([ var scrollTop = $parent.scrollTop(); // iterate over rendered mermaid charts $content.find('pre[data-plugin]:not([processed="true"])').each(function (index, el) { - var plugin = plugins[el.getAttribute('data-plugin')]; + var type = el.getAttribute('data-plugin'); + var plugin = plugins[type]; if (!plugin) { return; } // retrieve the attached source code which it was drawn @@ -666,7 +801,11 @@ define([ if (typeof(patch) === 'string') { throw new Error(patch); } else { - DD.apply($content[0], patch); + try { + DD.apply($content[0], patch); + } catch (err) { + console.error(err); + } var $mts = $content.find('media-tag'); $mts.each(function (i, el) { var $mt = $(el).contextmenu(function (e) { @@ -721,9 +860,35 @@ define([ if (target) { target.scrollIntoView(); } }); + // replace remote images with links to those images + $content.find('div.cp-inline-img-warning').each(function (index, el) { + if (!el) { return; } + var link = el.querySelector('a.cp-remote-img'); + if (!link) { return; } + link.onclick = function (ev) { + ev.preventDefault(); + ev.stopPropagation(); + common.openURL(link.href); + }; + }); + + // transform style tags into pre tags with the same content + // to be handled by the less rendering plugin + $content.find('style').each(function (index, el) { + var parent = el.parentElement; + var pre = h('pre', { + 'data-plugin': 'less', + 'less-src': canonicalizeLess(el.innerText), + style: 'display: none', + }, el.innerText); + parent.replaceChild(pre, el); + }); + // loop over plugin elements in the rendered content $content.find('pre[data-plugin]').each(function (index, el) { - var plugin = plugins[el.getAttribute('data-plugin')]; + var type = el.getAttribute('data-plugin'); + var plugin = plugins[type]; + if (!plugin) { return; } var $el = $(el); $el.off('contextmenu').on('contextmenu', function (e) { @@ -742,13 +907,17 @@ define([ // you can assume that the index of your rendered charts matches that // of those in the markdown source. var src = plugin.source[index]; - el.setAttribute(plugin.attr, src); + if (src) { + el.setAttribute(plugin.attr, src); + } var cached = getAvailableCachedElement($content, plugin.cache, src); // check if you had cached a pre-rendered instance of the supplied source if (typeof(cached) !== 'object') { try { - plugin.render($el); + plugin.render($el, { + scope: $content, + }); } catch (e) { console.error(e); } return; } diff --git a/www/common/drive-ui.js b/www/common/drive-ui.js index 4a75459a5..e079ad2b9 100644 --- a/www/common/drive-ui.js +++ b/www/common/drive-ui.js @@ -2571,14 +2571,7 @@ define([ var getNewPadTypes = function () { var arr = []; AppConfig.availablePadTypes.forEach(function (type) { - if (type === 'drive') { return; } - if (type === 'teams') { return; } - if (type === 'contacts') { return; } - if (type === 'todo') { return; } - if (type === 'file') { return; } - if (type === 'accounts') { return; } - if (type === 'calendar') { return; } - if (type === 'poll') { return; } // replaced by forms + if (AppConfig.hiddenTypes.indexOf(type) !== -1) { return; } if (!APP.loggedIn && AppConfig.registeredOnlyTypes && AppConfig.registeredOnlyTypes.indexOf(type) !== -1) { return; diff --git a/www/common/inner/common-mediatag.js b/www/common/inner/common-mediatag.js index 966724fa1..b8550fcac 100644 --- a/www/common/inner/common-mediatag.js +++ b/www/common/inner/common-mediatag.js @@ -127,9 +127,8 @@ define([ if (e || !data) { return void displayDefault(); } if (typeof data !== "number") { return void displayDefault(); } if (Util.bytesToMegabytes(data) > 0.5) { return void displayDefault(); } - var $img = $('').appendTo($container); - $img.attr('src', src); - $img.attr('data-crypto-key', 'cryptpad:' + cryptKey); + var mt = UI.mediaTag(src, cryptKey); + var $img = $(mt).appendTo($container); MT.displayMediatagImage(common, $img, function (err, $image) { if (err) { return void console.error(err); } centerImage($img, $image); diff --git a/www/common/notifications.js b/www/common/notifications.js index e6cad99e2..20738e237 100644 --- a/www/common/notifications.js +++ b/www/common/notifications.js @@ -8,7 +8,8 @@ define([ '/common/common-constants.js', '/customize/messages.js', '/customize/pages.js', -], function($, h, Hash, UI, UIElements, Util, Constants, Messages, Pages) { + '/lib/datepicker/flatpickr.js', +], function($, h, Hash, UI, UIElements, Util, Constants, Messages, Pages, Flatpickr) { var handlers = {}; @@ -477,7 +478,7 @@ define([ var nowDateStr = new Date().toLocaleDateString(); var startDate = new Date(start); if (msg.isAllDay && msg.startDay) { - startDate = new Date(msg.startDay); + startDate = Flatpickr.parseDate(msg.startDay); } // Missed events diff --git a/www/common/outer/async-store.js b/www/common/outer/async-store.js index 66e1967cd..9f779a0a8 100644 --- a/www/common/outer/async-store.js +++ b/www/common/outer/async-store.js @@ -274,9 +274,9 @@ define([ } var pads = data.pads || data; - s.rpc.pin(pads, function (e, hash) { + s.rpc.pin(pads, function (e) { if (e) { return void cb({error: e}); } - cb({hash: hash}); + cb({}); }); }; @@ -289,9 +289,9 @@ define([ if (!s.rpc) { return void cb({error: 'RPC_NOT_READY'}); } var pads = data.pads || data; - s.rpc.unpin(pads, function (e, hash) { + s.rpc.unpin(pads, function (e) { if (e) { return void cb({error: e}); } - cb({hash: hash}); + cb({}); }); }; @@ -394,9 +394,9 @@ define([ if (!store.rpc) { return void cb({error: 'RPC_NOT_READY'}); } var list = getCanonicalChannelList(false); - store.rpc.reset(list, function (e, hash) { + store.rpc.reset(list, function (e) { if (e) { return void cb(e); } - cb(null, hash); + cb(null); }); }; diff --git a/www/common/outer/team.js b/www/common/outer/team.js index 0ec224984..4184b8482 100644 --- a/www/common/outer/team.js +++ b/www/common/outer/team.js @@ -986,47 +986,60 @@ define([ var state = team.roster.getState() || {}; var members = state.members || {}; - // Get pending owners - var md = team.listmap.metadata || {}; - if (Array.isArray(md.pending_owners)) { - // Get the members associated to the pending_owners' edPublic and mark them as such - md.pending_owners.forEach(function (ed) { - var member; - Object.keys(members).some(function (curve) { - if (members[curve].edPublic === ed) { - member = members[curve]; - return true; - } - }); - if (!member && teamData.owner) { - var removeOwnership = function (chan) { - ctx.Store.setPadMetadata(null, { - channel: chan, - command: 'RM_PENDING_OWNERS', - value: [ed], - }, function () {}); - }; - removeOwnership(teamData.channel); - removeOwnership(Util.find(teamData, ['keys', 'roster', 'channel'])); - removeOwnership(Util.find(teamData, ['keys', 'chat', 'channel'])); + var md; + nThen(function (waitFor) { + // Get pending owners + ctx.Store.getPadMetadata(null, { + channel: teamData.channel + }, waitFor(function (obj) { + if (obj && obj.error) { + md = team.listmap.metadata || {}; return; } - member.pendingOwner = true; - }); - } + md = obj; + })); + }).nThen(function () { + ctx.pending_owners = md.pending_owners; + if (Array.isArray(md.pending_owners)) { + // Get the members associated to the pending_owners' edPublic and mark them as such + md.pending_owners.forEach(function (ed) { + var member; + Object.keys(members).some(function (curve) { + if (members[curve].edPublic === ed) { + member = members[curve]; + return true; + } + }); + if (!member && teamData.owner) { + var removeOwnership = function (chan) { + ctx.Store.setPadMetadata(null, { + channel: chan, + command: 'RM_PENDING_OWNERS', + value: [ed], + }, function () {}); + }; + removeOwnership(teamData.channel); + removeOwnership(Util.find(teamData, ['keys', 'roster', 'channel'])); + removeOwnership(Util.find(teamData, ['keys', 'chat', 'channel'])); + return; + } + member.pendingOwner = true; + }); + } - // Add online status (using messenger data) - if (ctx.store.messenger) { - var chatData = team.getChatData(); - var online = ctx.store.messenger.getOnlineList(chatData.channel) || []; - online.forEach(function (curve) { - if (members[curve]) { - members[curve].online = true; - } - }); - } + // Add online status (using messenger data) + if (ctx.store.messenger) { + var chatData = team.getChatData(); + var online = ctx.store.messenger.getOnlineList(chatData.channel) || []; + online.forEach(function (curve) { + if (members[curve]) { + members[curve].online = true; + } + }); + } - cb(members); + cb(members); + }); }; // Return folders with edit rights available to everybody (decrypted pad href) @@ -1144,8 +1157,7 @@ define([ if (!teamData) { return void cb ({error: 'ENOENT'}); } var team = ctx.teams[teamId]; if (!team) { return void cb ({error: 'ENOENT'}); } - var md = team.listmap.metadata || {}; - var isPendingOwner = (md.pending_owners || []).indexOf(user.edPublic) !== -1; + var isPendingOwner = user.pendingOwner; nThen(function (waitFor) { var cmd = isPendingOwner ? 'RM_PENDING_OWNERS' : 'RM_OWNERS'; @@ -1364,42 +1376,60 @@ define([ var describeUser = function (ctx, data, cId, cb) { var teamId = data.teamId; if (!teamId) { return void cb({error: 'EINVAL'}); } + var teamData = Util.find(ctx, ['store', 'proxy', 'teams', teamId]); var team = ctx.teams[teamId]; - if (!team) { return void cb ({error: 'ENOENT'}); } + if (!teamData || !team) { return void cb ({error: 'ENOENT'}); } if (!team.roster) { return void cb({error: 'NO_ROSTER'}); } if (!data.curvePublic || !data.data) { return void cb({error: 'MISSING_DATA'}); } var state = team.roster.getState(); var user = state.members[data.curvePublic]; - // It it is an ownership revocation, we have to set it in pad metadata first - if (user.role === "OWNER" && data.data.role !== "OWNER") { - revokeOwnership(ctx, teamId, user, function (err) { - if (!err) { return void cb(); } - console.error(err); - return void cb({error: err}); - }); - return; - } + var md; + nThen(function (waitFor) { + // Get pending owners + ctx.Store.getPadMetadata(null, { + channel: teamData.channel + }, waitFor(function (obj) { + if (obj && obj.error) { + md = team.listmap.metadata || {}; + return; + } + md = obj; + })); + }).nThen(function () { + user.pendingOwner = Array.isArray(md.pending_owners) && + md.pending_owners.indexOf(user.edPublic) !== -1; - // Viewer to editor - if (user.role === "VIEWER" && data.data.role !== "VIEWER") { - changeEditRights(ctx, teamId, user, true, function (obj) { - return void cb(obj); - }); - } + // It it is an ownership revocation, we have to set it in pad metadata first + if (user.role === "OWNER" && data.data.role !== "OWNER") { + revokeOwnership(ctx, teamId, user, function (err) { + if (!err) { return void cb(); } + console.error(err); + return void cb({error: err}); + }); + return; + } - // Editor to viewer - if (user.role !== "VIEWER" && data.data.role === "VIEWER") { - changeEditRights(ctx, teamId, user, false, function (obj) { - return void cb(obj); - }); - } + // Viewer to editor + if (user.role === "VIEWER" && data.data.role !== "VIEWER") { + changeEditRights(ctx, teamId, user, true, function (obj) { + return void cb(obj); + }); + } - var obj = {}; - obj[data.curvePublic] = data.data; - team.roster.describe(obj, function (err) { - if (err) { return void cb({error: err}); } - cb(); + // Editor to viewer + if (user.role !== "VIEWER" && data.data.role === "VIEWER") { + changeEditRights(ctx, teamId, user, false, function (obj) { + return void cb(obj); + }); + } + + var obj = {}; + obj[data.curvePublic] = data.data; + team.roster.describe(obj, function (err) { + if (err) { return void cb({error: err}); } + cb(); + }); }); }; @@ -2010,9 +2040,16 @@ define([ if (['drive', 'teams', 'settings'].indexOf(app) !== -1) { safe = true; } Object.keys(teams).forEach(function (id) { if (!ctx.teams[id]) { return; } + var proxy = ctx.teams[id].proxy || {}; + var nPads = proxy.drive && Object.keys(proxy.drive.filesData || {}).length; + var nSf = proxy.drive && Object.keys(proxy.drive.sharedFolders || {}).length; t[id] = { owner: teams[id].owner, name: teams[id].metadata.name, + channel: teams[id].channel, + numberPads: nPads, + numberSf: nSf, + roster: Util.find(teams[id], ['keys', 'roster', 'channel']), edPublic: Util.find(teams[id], ['keys', 'drive', 'edPublic']), avatar: Util.find(teams[id], ['metadata', 'avatar']), viewer: !Util.find(teams[id], ['keys', 'drive', 'edPrivate']), diff --git a/www/common/pinpad.js b/www/common/pinpad.js index 7e9cd4ee2..de93f066f 100644 --- a/www/common/pinpad.js +++ b/www/common/pinpad.js @@ -26,23 +26,19 @@ var factory = function (Util, Rpc) { exp.send = rpc.send; // you can ask the server to pin a particular channel for you - exp.pin = function (channels, cb) { + exp.pin = function (channels, _cb) { + var cb = Util.once(Util.mkAsync(_cb)); if (!Array.isArray(channels)) { - setTimeout(function () { - cb('[TypeError] pin expects an array'); - }); - return; + return void cb('[TypeError] pin expects an array'); } rpc.send('PIN', channels, cb); }; // you can also ask to unpin a particular channel - exp.unpin = function (channels, cb) { + exp.unpin = function (channels, _cb) { + var cb = Util.once(Util.mkAsync(_cb)); if (!Array.isArray(channels)) { - setTimeout(function () { - cb('[TypeError] pin expects an array'); - }); - return; + return void cb('[TypeError] pin expects an array'); } rpc.send('UNPIN', channels, cb); }; @@ -70,23 +66,12 @@ var factory = function (Util, Rpc) { }; // if local and remote hashes don't match, send a reset - exp.reset = function (channels, cb) { + exp.reset = function (channels, _cb) { + var cb = Util.once(Util.mkAsync(_cb)); if (!Array.isArray(channels)) { - setTimeout(function () { - cb('[TypeError] pin expects an array'); - }); - return; + return void cb('[TypeError] pin expects an array'); } - rpc.send('RESET', channels, function (e, response) { - if (e) { - return void cb(e); - } - if (!response.length) { - console.log(response); - return void cb('INVALID_RESPONSE'); - } - cb(e, response[0]); - }); + rpc.send('RESET', channels, cb); }; // get the combined size of all channels (in bytes) for all the diff --git a/www/common/sframe-app-framework.js b/www/common/sframe-app-framework.js index 796640f6d..0fd250f80 100644 --- a/www/common/sframe-app-framework.js +++ b/www/common/sframe-app-framework.js @@ -748,8 +748,8 @@ define([ var privateDat = cpNfInner.metadataMgr.getPrivateData(); var origin = privateDat.fileHost || privateDat.origin; var src = data.src = data.src.slice(0,1) === '/' ? origin + data.src : data.src; - mediaTagEmbedder($(''), data); + var mt = UI.mediaTag(src, data.key); + mediaTagEmbedder($(mt), data); }); }).appendTo(toolbar.$bottomL).hide(); }; diff --git a/www/common/sframe-common.js b/www/common/sframe-common.js index 3fd4869d1..5d3738cbb 100644 --- a/www/common/sframe-common.js +++ b/www/common/sframe-common.js @@ -145,8 +145,7 @@ define([ var hexFileName = secret.channel; var origin = data.fileHost || data.origin; var src = origin + Hash.getBlobPathFromHex(hexFileName); - return '' + - ''; + return UI.mediaTag(src, key).outerHTML; } return; }; diff --git a/www/convert/app-convert.less b/www/convert/app-convert.less new file mode 100644 index 000000000..c5813641a --- /dev/null +++ b/www/convert/app-convert.less @@ -0,0 +1,16 @@ +@import (reference) '../../customize/src/less2/include/framework.less'; +@import (reference) '../../customize/src/less2/include/sidebar-layout.less'; + +&.cp-app-convert { + + .framework_min_main( + @bg-color: @colortheme_apps[default], + ); + .sidebar-layout_main(); + + // body + display: flex; + flex-flow: column; + background-color: @cp_app-bg; + +} diff --git a/www/convert/index.html b/www/convert/index.html new file mode 100644 index 000000000..96a3cce86 --- /dev/null +++ b/www/convert/index.html @@ -0,0 +1,12 @@ + + + + CryptPad + + + + + + + + diff --git a/www/convert/inner.html b/www/convert/inner.html new file mode 100644 index 000000000..206b85722 --- /dev/null +++ b/www/convert/inner.html @@ -0,0 +1,18 @@ + + + + + + + + +
    +
    + + + diff --git a/www/convert/inner.js b/www/convert/inner.js new file mode 100644 index 000000000..cdcf745ca --- /dev/null +++ b/www/convert/inner.js @@ -0,0 +1,258 @@ +define([ + 'jquery', + '/api/config', + '/bower_components/chainpad-crypto/crypto.js', + '/common/toolbar.js', + '/bower_components/nthen/index.js', + '/common/sframe-common.js', + '/common/hyperscript.js', + '/customize/messages.js', + '/common/common-interface.js', + '/common/common-util.js', + + '/bower_components/file-saver/FileSaver.min.js', + 'css!/bower_components/bootstrap/dist/css/bootstrap.min.css', + 'css!/bower_components/components-font-awesome/css/font-awesome.min.css', + 'less!/convert/app-convert.less', +], function ( + $, + ApiConfig, + Crypto, + Toolbar, + nThen, + SFCommon, + h, + Messages, + UI, + Util + ) +{ + var APP = {}; + + var common; + var sFrameChan; + + var debug = console.debug; + + var x2tReady = Util.mkEvent(true); + var x2tInitialized = false; + var x2tInit = function(x2t) { + debug("x2t mount"); + // x2t.FS.mount(x2t.MEMFS, {} , '/'); + x2t.FS.mkdir('/working'); + x2t.FS.mkdir('/working/media'); + x2t.FS.mkdir('/working/fonts'); + x2tInitialized = true; + x2tReady.fire(); + //fetchFonts(x2t); + debug("x2t mount done"); + }; + var getX2t = function (cb) { + require(['/common/onlyoffice/x2t/x2t.js'], function() { // FIXME why does this fail without an access-control-allow-origin header? + var x2t = window.Module; + x2t.run(); + if (x2tInitialized) { + debug("x2t runtime already initialized"); + return void x2tReady.reg(function () { + cb(x2t); + }); + } + + x2t.onRuntimeInitialized = function() { + debug("x2t in runtime initialized"); + // Init x2t js module + x2tInit(x2t); + x2tReady.reg(function () { + cb(x2t); + }); + }; + }); + }; + /* + Converting Data + + This function converts a data in a specific format to the outputformat + The filename extension needs to represent the input format + Example: fileName=cryptpad.bin outputFormat=xlsx + */ + var getFormatId = function (ext) { + // Sheets + if (ext === 'xlsx') { return 257; } + if (ext === 'xls') { return 258; } + if (ext === 'ods') { return 259; } + if (ext === 'csv') { return 260; } + if (ext === 'pdf') { return 513; } + // Docs + if (ext === 'docx') { return 65; } + if (ext === 'doc') { return 66; } + if (ext === 'odt') { return 67; } + if (ext === 'txt') { return 69; } + if (ext === 'html') { return 70; } + + // Slides + if (ext === 'pptx') { return 129; } + if (ext === 'ppt') { return 130; } + if (ext === 'odp') { return 131; } + + return; + }; + var getFromId = function (ext) { + var id = getFormatId(ext); + if (!id) { return ''; } + return ''+id+''; + }; + var getToId = function (ext) { + var id = getFormatId(ext); + if (!id) { return ''; } + return ''+id+''; + }; + var x2tConvertDataInternal = function(x2t, data, fileName, outputFormat) { + debug("Converting Data for " + fileName + " to " + outputFormat); + + var inputFormat = fileName.split('.').pop(); + + x2t.FS.writeFile('/working/' + fileName, data); + var params = "" + + "" + + "/working/" + fileName + "" + + "/working/" + fileName + "." + outputFormat + "" + + getFromId(inputFormat) + + getToId(outputFormat) + + "false" + + ""; + // writing params file to mounted working disk (in memory) + x2t.FS.writeFile('/working/params.xml', params); + // running conversion + x2t.ccall("runX2T", ["number"], ["string"], ["/working/params.xml"]); + // reading output file from working disk (in memory) + var result; + try { + result = x2t.FS.readFile('/working/' + fileName + "." + outputFormat); + } catch (e) { + console.error(e, x2t.FS); + debug("Failed reading converted file"); + UI.warn(Messages.error); + return ""; + } + return result; + }; + var x2tConverter = function (typeSrc, typeTarget) { + return function (data, name, cb) { + getX2t(function (x2t) { + if (typeSrc === 'ods') { + data = x2tConvertDataInternal(x2t, data, name, 'xlsx'); + name += '.xlsx'; + } + if (typeSrc === 'odt') { + data = x2tConvertDataInternal(x2t, data, name, 'docx'); + name += '.docx'; + } + if (typeSrc === 'odp') { + data = x2tConvertDataInternal(x2t, data, name, 'pptx'); + name += '.pptx'; + } + cb(x2tConvertDataInternal(x2t, data, name, typeTarget)); + }); + }; + }; + + var CONVERTERS = { + xlsx: { + //pdf: x2tConverter('xlsx', 'pdf'), + ods: x2tConverter('xlsx', 'ods'), + bin: x2tConverter('xlsx', 'bin'), + }, + ods: { + //pdf: x2tConverter('ods', 'pdf'), + xlsx: x2tConverter('ods', 'xlsx'), + bin: x2tConverter('ods', 'bin'), + }, + odt: { + docx: x2tConverter('odt', 'docx'), + txt: x2tConverter('odt', 'txt'), + bin: x2tConverter('odt', 'bin'), + }, + docx: { + odt: x2tConverter('docx', 'odt'), + txt: x2tConverter('docx', 'txt'), + bin: x2tConverter('docx', 'bin'), + }, + txt: { + odt: x2tConverter('txt', 'odt'), + docx: x2tConverter('txt', 'docx'), + bin: x2tConverter('txt', 'bin'), + }, + odp: { + pptx: x2tConverter('odp', 'pptx'), + bin: x2tConverter('odp', 'bin'), + }, + pptx: { + odp: x2tConverter('pptx', 'odp'), + bin: x2tConverter('pptx', 'bin'), + }, + }; + + Messages.convertPage = "Convert"; // XXX + Messages.convert_hint = "Pick the file you want to convert. The list of output format will be visible afterward."; // XXX + + var createToolbar = function () { + var displayed = ['useradmin', 'newpad', 'limit', 'pageTitle', 'notifications']; + var configTb = { + displayed: displayed, + sfCommon: common, + $container: APP.$toolbar, + pageTitle: Messages.convertPage, + metadataMgr: common.getMetadataMgr(), + }; + APP.toolbar = Toolbar.create(configTb); + APP.toolbar.$rightside.hide(); + }; + + nThen(function (waitFor) { + $(waitFor(UI.addLoadingScreen)); + SFCommon.create(waitFor(function (c) { APP.common = common = c; })); + }).nThen(function (waitFor) { + APP.$container = $('#cp-sidebarlayout-container'); + APP.$toolbar = $('#cp-toolbar'); + APP.$leftside = $('
    ', {id: 'cp-sidebarlayout-leftside'}).appendTo(APP.$container); + APP.$rightside = $('
    ', {id: 'cp-sidebarlayout-rightside'}).appendTo(APP.$container); + sFrameChan = common.getSframeChannel(); + sFrameChan.onReady(waitFor()); + }).nThen(function (/*waitFor*/) { + createToolbar(); + + var hint = h('p.cp-convert-hint', Messages.convert_hint); + + var picker = h('input', { + type: 'file' + }); + APP.$rightside.append([hint, picker]); + + $(picker).on('change', function () { + var file = picker.files[0]; + var name = file && file.name; + var reader = new FileReader(); + var parsed = file && file.name && /.+\.([^.]+)$/.exec(file.name); + var ext = parsed && parsed[1]; + reader.onload = function (e) { + if (CONVERTERS[ext]) { + Object.keys(CONVERTERS[ext]).forEach(function (to) { + var button = h('button.btn', to); + $(button).click(function () { + CONVERTERS[ext][to](new Uint8Array(e.target.result), name, function (a) { + var n = name.slice(0, -ext.length) + to; + var blob = new Blob([a], {type: "application/bin;charset=utf-8"}); + window.saveAs(blob, n); + }); + + }).appendTo(APP.$rightside); + }); + } + }; + reader.readAsArrayBuffer(file, 'application/octet-stream'); + }); + + UI.removeLoadingScreen(); + + }); +}); diff --git a/www/convert/main.js b/www/convert/main.js new file mode 100644 index 000000000..236290b13 --- /dev/null +++ b/www/convert/main.js @@ -0,0 +1,28 @@ +// Load #1, load as little as possible because we are in a race to get the loading screen up. +define([ + '/bower_components/nthen/index.js', + '/api/config', + '/common/dom-ready.js', + '/common/sframe-common-outer.js' +], function (nThen, ApiConfig, DomReady, SFCommonO) { + + // Loaded in load #2 + nThen(function (waitFor) { + DomReady.onReady(waitFor()); + }).nThen(function (waitFor) { + SFCommonO.initIframe(waitFor, true); + }).nThen(function (/*waitFor*/) { + var category; + if (window.location.hash) { + category = window.location.hash.slice(1); + window.location.hash = ''; + } + var addData = function (obj) { + if (category) { obj.category = category; } + }; + SFCommonO.start({ + noRealtime: true, + addData: addData + }); + }); +}); diff --git a/www/form/app-form.less b/www/form/app-form.less index 433316448..95802e18b 100644 --- a/www/form/app-form.less +++ b/www/form/app-form.less @@ -470,7 +470,11 @@ //padding: 10px; } + .cp-form-creator-results-export { + margin-bottom: 20px; + } .cp-form-creator-results-content { + padding-bottom: 100px; .cp-form-block { background: @cp_form-bg1; padding: 10px; diff --git a/www/form/export.js b/www/form/export.js new file mode 100644 index 000000000..4afc4170a --- /dev/null +++ b/www/form/export.js @@ -0,0 +1,72 @@ +define([ + '/common/common-util.js', + '/customize/messages.js' +], function (Util, Messages) { + var Export = {}; + + var escapeCSV = function (v) { + if (!/("|,|\n|;)/.test(v)) { + return v || ''; + } + var value = ''; + var vv = (v || '').replaceAll('"', '""'); + value += '"' + vv + '"'; + return value; + }; + Export.results = function (content, answers, TYPES) { + if (!content || !content.form) { return; } + var csv = ""; + var form = content.form; + + var questions = [Messages.form_poll_time, Messages.share_formView]; + + content.order.forEach(function (key) { + var obj = form[key]; + if (!obj) { return; } + var type = obj.type; + if (!TYPES[type]) { return; } // Ignore static types + var c; + if (TYPES[type] && TYPES[type].exportCSV) { c = TYPES[type].exportCSV(false, obj); } + if (!c) { c = [obj.q || Messages.form_default]; } + Array.prototype.push.apply(questions, c); + }); + + questions.forEach(function (v, i) { + if (i) { csv += ','; } + csv += escapeCSV(v); + }); + + Object.keys(answers || {}).forEach(function (key) { + var obj = answers[key]; + csv += '\n'; + var time = new Date(obj.time).toISOString(); + var msg = obj.msg || {}; + var user = msg._userdata || {}; + csv += escapeCSV(time); + csv += ',' + escapeCSV(user.name || Messages.anonymous); + content.order.forEach(function (key) { + var type = form[key].type; + if (!TYPES[type]) { return; } // Ignore static types + if (TYPES[type].exportCSV) { + var res = TYPES[type].exportCSV(msg[key], form[key]).map(function (str) { + return escapeCSV(str); + }).join(','); + csv += ',' + res; + return; + } + csv += ',' + escapeCSV(String(msg[key] || '')); + }); + }); + return csv; + }; + + Export.main = function (content, cb) { + var json = Util.clone(content || {}); + delete json.answers; + cb(new Blob([JSON.stringify(json, 0, 2)], { + type: 'application/json;charset=utf-8' + })); + }; + + return Export; +}); diff --git a/www/form/inner.js b/www/form/inner.js index 80c50a0d6..2a1501ecb 100644 --- a/www/form/inner.js +++ b/www/form/inner.js @@ -4,6 +4,7 @@ define([ '/bower_components/chainpad-crypto/crypto.js', '/common/sframe-app-framework.js', '/common/toolbar.js', + '/form/export.js', '/bower_components/nthen/index.js', '/common/sframe-common.js', '/common/common-util.js', @@ -30,6 +31,8 @@ define([ 'cm/mode/gfm/gfm', 'css!cm/lib/codemirror.css', + '/bower_components/file-saver/FileSaver.min.js', + 'css!/bower_components/codemirror/lib/codemirror.css', 'css!/bower_components/codemirror/addon/dialog/dialog.css', 'css!/bower_components/codemirror/addon/fold/foldgutter.css', @@ -42,6 +45,7 @@ define([ Crypto, Framework, Toolbar, + Exporter, nThen, SFCommon, Util, @@ -76,8 +80,11 @@ define([ timeFormat = "h:i K"; } - var MAX_OPTIONS = 15; // XXX - var MAX_ITEMS = 10; // XXX + // multi-line radio, checkboxes, and possibly other things have a max number of items + // we'll consider increasing this restriction if people are unhappy with it + // but as a general rule we expect users will appreciate having simpler questions + var MAX_OPTIONS = 15; + var MAX_ITEMS = 10; var saveAndCancelOptions = function (getRes, cb) { // Cancel changes @@ -1208,6 +1215,20 @@ define([ return h('div.cp-form-results-type-radio', results); }, + exportCSV: function (answer, form) { + var opts = form.opts || {}; + var q = form.q || Messages.form_default; + if (answer === false) { + return (opts.items || []).map(function (obj) { + return q + ' | ' + obj.v; + }); + } + if (!answer) { return ['']; } + return (opts.items || []).map(function (obj) { + var uid = obj.uid; + return String(answer[uid] || ''); + }); + }, icon: h('i.cptools.cptools-form-grid-radio') }, checkbox: { @@ -1422,6 +1443,20 @@ define([ return h('div.cp-form-results-type-radio', results); }, + exportCSV: function (answer, form) { + var opts = form.opts || {}; + var q = form.q || Messages.form_default; + if (answer === false) { + return (opts.items || []).map(function (obj) { + return q + ' | ' + obj.v; + }); + } + if (!answer) { return ['']; } + return (opts.items || []).map(function (obj) { + var uid = obj.uid; + return String(answer[uid] || ''); + }); + }, icon: h('i.cptools.cptools-form-grid-check') }, sort: { @@ -1642,6 +1677,16 @@ define([ return h('div.cp-form-type-poll', lines); }, + exportCSV: function (answer) { + if (answer === false) { return; } + if (!answer || !answer.values) { return ['']; } + var str = ''; + Object.keys(answer.values).sort().forEach(function (k, i) { + if (i !== 0) { str += ';'; } + str += k.replace(';', '').replace(':', '') + ':' + answer.values[k]; + }); + return [str]; + }, icon: h('i.cptools.cptools-form-poll') }, }; @@ -1656,9 +1701,21 @@ define([ var controls = h('div.cp-form-creator-results-controls'); var $controls = $(controls).appendTo($container); + var exportButton = h('button.btn.btn-secondary', Messages.exportButton); // XXX form_exportCSV; + var exportCSV = h('div.cp-form-creator-results-export', exportButton); + $(exportCSV).appendTo($container); var results = h('div.cp-form-creator-results-content'); var $results = $(results).appendTo($container); + $(exportButton).click(function () { + var csv = Exporter.results(content, answers, TYPES); + if (!csv) { return void UI.warn(Messages.error); } + var suggestion = APP.framework._.title.suggestTitle('cryptpad-document'); + var title = Util.fixFileName(suggestion) + '.csv'; + window.saveAs(new Blob([csv], { + type: 'text/csv' + }), title); + }); var summary = true; var form = content.form; @@ -2315,6 +2372,7 @@ define([ var andThen = function (framework) { framework.start(); + APP.framework = framework; var evOnChange = Util.mkEvent(); var content = {}; @@ -2739,6 +2797,17 @@ define([ return content; }); + framework.setFileImporter({ accept: ['.json'] }, function (newContent) { + var parsed = JSON.parse(newContent || {}); + parsed.answers = content.answers; + return parsed; + }); + + framework.setFileExporter(['.json'], function(cb, ext) { + Exporter.main(content, cb, ext); + }, true); + + }; Framework.create({ diff --git a/www/form/templates.js b/www/form/templates.js new file mode 100644 index 000000000..6ff3249b8 --- /dev/null +++ b/www/form/templates.js @@ -0,0 +1,20 @@ +define([ + '/customize/messages.js' +], function (Messages) { + return [{ + id: 'a', + used: 1, + name: Messages.form_type_poll, + content: { + form: { + "1": { + type: 'md' + }, + "2": { + type: 'poll' + } + }, + order: ["1", "2"] + } + }]; +}); diff --git a/www/kanban/export.js b/www/kanban/export.js index 9ee770ac7..694724cf4 100644 --- a/www/kanban/export.js +++ b/www/kanban/export.js @@ -13,6 +13,73 @@ define([ })); }; + module.import = function (content) { + // Import from Trello + + var c = { + data: {}, + items: {}, + list: [] + }; + + var colorMap = { + red: 'color1', + orange: 'color2', + yellow: 'color3', + lime: 'color4', + green: 'color5', + sky: 'color6', + blue: 'color7', + purple: 'color8', + pink: 'color9', + black: 'nocolor' + }; + content.cards.forEach(function (obj, i) { + var tags; + var color; + if (Array.isArray(obj.labels)) { + obj.labels.forEach(function (l) { + if (!color) { + color = colorMap[l.color] || ''; + } + if (l.name) { + tags = tags || []; + var n = l.name.toLowerCase().trim(); + if (tags.indexOf(n) === -1) { tags.push(n); } + } + }); + } + c.items[(i+1)] = { + id: (i+1), + title: obj.name, + body: obj.desc, + color: color, + tags: tags + }; + }); + + var id = 1; + content.lists.forEach(function (obj) { + var _id = obj.id; + var cards = []; + content.cards.forEach(function (card, i) { + if (card.idList === _id) { + cards.push(i+1); + } + }); + c.data[id] = { + id: id, + title: obj.name, + item: cards + }; + c.list.push(id); + + id++; + }); + + return c; + }; + return module; }); diff --git a/www/kanban/inner.js b/www/kanban/inner.js index 76a9fe1cc..4832adcca 100644 --- a/www/kanban/inner.js +++ b/www/kanban/inner.js @@ -18,6 +18,7 @@ define([ '/bower_components/marked/marked.min.js', 'cm/lib/codemirror', '/kanban/jkanban_cp.js', + '/kanban/export.js', 'cm/mode/gfm/gfm', 'cm/addon/edit/closebrackets', @@ -50,7 +51,8 @@ define([ ChainPad, Marked, CodeMirror, - jKanban) + jKanban, + Export) { var verbose = function (x) { console.log(x); }; @@ -287,7 +289,7 @@ define([ var fileHost = privateData.fileHost || privateData.origin; var src = fileHost + Hash.getBlobPathFromHex(secret.channel); var key = Hash.encodeBase64(secret.keys.cryptKey); - var mt = ''; + var mt = UI.mediaTag(src, key).outerHTML; editor.replaceSelection(mt); } }; @@ -1060,6 +1062,11 @@ define([ var parsed; try { parsed = JSON.parse(content); } catch (e) { return void console.error(e); } + + if (parsed && parsed.id && parsed.lists && parsed.cards) { + return { content: Export.import(parsed) }; + } + return { content: parsed }; }); diff --git a/www/lib/changelog.md b/www/lib/changelog.md new file mode 100644 index 000000000..86ef6ec22 --- /dev/null +++ b/www/lib/changelog.md @@ -0,0 +1,4 @@ +This file is intended to be used as a log of what third-party source we have vendored, where we got it, and what modifications we have made to it (if any). + +* [turndown v7.1.1](https://github.com/mixmark-io/turndown/releases/tag/v7.1.1) built from unmodified source as per its build scripts. + diff --git a/www/lib/turndown.browser.umd.js b/www/lib/turndown.browser.umd.js new file mode 100644 index 000000000..a812101ae --- /dev/null +++ b/www/lib/turndown.browser.umd.js @@ -0,0 +1,976 @@ +(function (global, factory) { + typeof exports === 'object' && typeof module !== 'undefined' ? module.exports = factory() : + typeof define === 'function' && define.amd ? define(factory) : + (global = typeof globalThis !== 'undefined' ? globalThis : global || self, global.TurndownService = factory()); +}(this, (function () { 'use strict'; + + function extend (destination) { + for (var i = 1; i < arguments.length; i++) { + var source = arguments[i]; + for (var key in source) { + if (source.hasOwnProperty(key)) destination[key] = source[key]; + } + } + return destination + } + + function repeat (character, count) { + return Array(count + 1).join(character) + } + + function trimLeadingNewlines (string) { + return string.replace(/^\n*/, '') + } + + function trimTrailingNewlines (string) { + // avoid match-at-end regexp bottleneck, see #370 + var indexEnd = string.length; + while (indexEnd > 0 && string[indexEnd - 1] === '\n') indexEnd--; + return string.substring(0, indexEnd) + } + + var blockElements = [ + 'ADDRESS', 'ARTICLE', 'ASIDE', 'AUDIO', 'BLOCKQUOTE', 'BODY', 'CANVAS', + 'CENTER', 'DD', 'DIR', 'DIV', 'DL', 'DT', 'FIELDSET', 'FIGCAPTION', 'FIGURE', + 'FOOTER', 'FORM', 'FRAMESET', 'H1', 'H2', 'H3', 'H4', 'H5', 'H6', 'HEADER', + 'HGROUP', 'HR', 'HTML', 'ISINDEX', 'LI', 'MAIN', 'MENU', 'NAV', 'NOFRAMES', + 'NOSCRIPT', 'OL', 'OUTPUT', 'P', 'PRE', 'SECTION', 'TABLE', 'TBODY', 'TD', + 'TFOOT', 'TH', 'THEAD', 'TR', 'UL' + ]; + + function isBlock (node) { + return is(node, blockElements) + } + + var voidElements = [ + 'AREA', 'BASE', 'BR', 'COL', 'COMMAND', 'EMBED', 'HR', 'IMG', 'INPUT', + 'KEYGEN', 'LINK', 'META', 'PARAM', 'SOURCE', 'TRACK', 'WBR' + ]; + + function isVoid (node) { + return is(node, voidElements) + } + + function hasVoid (node) { + return has(node, voidElements) + } + + var meaningfulWhenBlankElements = [ + 'A', 'TABLE', 'THEAD', 'TBODY', 'TFOOT', 'TH', 'TD', 'IFRAME', 'SCRIPT', + 'AUDIO', 'VIDEO' + ]; + + function isMeaningfulWhenBlank (node) { + return is(node, meaningfulWhenBlankElements) + } + + function hasMeaningfulWhenBlank (node) { + return has(node, meaningfulWhenBlankElements) + } + + function is (node, tagNames) { + return tagNames.indexOf(node.nodeName) >= 0 + } + + function has (node, tagNames) { + return ( + node.getElementsByTagName && + tagNames.some(function (tagName) { + return node.getElementsByTagName(tagName).length + }) + ) + } + + var rules = {}; + + rules.paragraph = { + filter: 'p', + + replacement: function (content) { + return '\n\n' + content + '\n\n' + } + }; + + rules.lineBreak = { + filter: 'br', + + replacement: function (content, node, options) { + return options.br + '\n' + } + }; + + rules.heading = { + filter: ['h1', 'h2', 'h3', 'h4', 'h5', 'h6'], + + replacement: function (content, node, options) { + var hLevel = Number(node.nodeName.charAt(1)); + + if (options.headingStyle === 'setext' && hLevel < 3) { + var underline = repeat((hLevel === 1 ? '=' : '-'), content.length); + return ( + '\n\n' + content + '\n' + underline + '\n\n' + ) + } else { + return '\n\n' + repeat('#', hLevel) + ' ' + content + '\n\n' + } + } + }; + + rules.blockquote = { + filter: 'blockquote', + + replacement: function (content) { + content = content.replace(/^\n+|\n+$/g, ''); + content = content.replace(/^/gm, '> '); + return '\n\n' + content + '\n\n' + } + }; + + rules.list = { + filter: ['ul', 'ol'], + + replacement: function (content, node) { + var parent = node.parentNode; + if (parent.nodeName === 'LI' && parent.lastElementChild === node) { + return '\n' + content + } else { + return '\n\n' + content + '\n\n' + } + } + }; + + rules.listItem = { + filter: 'li', + + replacement: function (content, node, options) { + content = content + .replace(/^\n+/, '') // remove leading newlines + .replace(/\n+$/, '\n') // replace trailing newlines with just a single one + .replace(/\n/gm, '\n '); // indent + var prefix = options.bulletListMarker + ' '; + var parent = node.parentNode; + if (parent.nodeName === 'OL') { + var start = parent.getAttribute('start'); + var index = Array.prototype.indexOf.call(parent.children, node); + prefix = (start ? Number(start) + index : index + 1) + '. '; + } + return ( + prefix + content + (node.nextSibling && !/\n$/.test(content) ? '\n' : '') + ) + } + }; + + rules.indentedCodeBlock = { + filter: function (node, options) { + return ( + options.codeBlockStyle === 'indented' && + node.nodeName === 'PRE' && + node.firstChild && + node.firstChild.nodeName === 'CODE' + ) + }, + + replacement: function (content, node, options) { + return ( + '\n\n ' + + node.firstChild.textContent.replace(/\n/g, '\n ') + + '\n\n' + ) + } + }; + + rules.fencedCodeBlock = { + filter: function (node, options) { + return ( + options.codeBlockStyle === 'fenced' && + node.nodeName === 'PRE' && + node.firstChild && + node.firstChild.nodeName === 'CODE' + ) + }, + + replacement: function (content, node, options) { + var className = node.firstChild.getAttribute('class') || ''; + var language = (className.match(/language-(\S+)/) || [null, ''])[1]; + var code = node.firstChild.textContent; + + var fenceChar = options.fence.charAt(0); + var fenceSize = 3; + var fenceInCodeRegex = new RegExp('^' + fenceChar + '{3,}', 'gm'); + + var match; + while ((match = fenceInCodeRegex.exec(code))) { + if (match[0].length >= fenceSize) { + fenceSize = match[0].length + 1; + } + } + + var fence = repeat(fenceChar, fenceSize); + + return ( + '\n\n' + fence + language + '\n' + + code.replace(/\n$/, '') + + '\n' + fence + '\n\n' + ) + } + }; + + rules.horizontalRule = { + filter: 'hr', + + replacement: function (content, node, options) { + return '\n\n' + options.hr + '\n\n' + } + }; + + rules.inlineLink = { + filter: function (node, options) { + return ( + options.linkStyle === 'inlined' && + node.nodeName === 'A' && + node.getAttribute('href') + ) + }, + + replacement: function (content, node) { + var href = node.getAttribute('href'); + var title = cleanAttribute(node.getAttribute('title')); + if (title) title = ' "' + title + '"'; + return '[' + content + '](' + href + title + ')' + } + }; + + rules.referenceLink = { + filter: function (node, options) { + return ( + options.linkStyle === 'referenced' && + node.nodeName === 'A' && + node.getAttribute('href') + ) + }, + + replacement: function (content, node, options) { + var href = node.getAttribute('href'); + var title = cleanAttribute(node.getAttribute('title')); + if (title) title = ' "' + title + '"'; + var replacement; + var reference; + + switch (options.linkReferenceStyle) { + case 'collapsed': + replacement = '[' + content + '][]'; + reference = '[' + content + ']: ' + href + title; + break + case 'shortcut': + replacement = '[' + content + ']'; + reference = '[' + content + ']: ' + href + title; + break + default: + var id = this.references.length + 1; + replacement = '[' + content + '][' + id + ']'; + reference = '[' + id + ']: ' + href + title; + } + + this.references.push(reference); + return replacement + }, + + references: [], + + append: function (options) { + var references = ''; + if (this.references.length) { + references = '\n\n' + this.references.join('\n') + '\n\n'; + this.references = []; // Reset references + } + return references + } + }; + + rules.emphasis = { + filter: ['em', 'i'], + + replacement: function (content, node, options) { + if (!content.trim()) return '' + return options.emDelimiter + content + options.emDelimiter + } + }; + + rules.strong = { + filter: ['strong', 'b'], + + replacement: function (content, node, options) { + if (!content.trim()) return '' + return options.strongDelimiter + content + options.strongDelimiter + } + }; + + rules.code = { + filter: function (node) { + var hasSiblings = node.previousSibling || node.nextSibling; + var isCodeBlock = node.parentNode.nodeName === 'PRE' && !hasSiblings; + + return node.nodeName === 'CODE' && !isCodeBlock + }, + + replacement: function (content) { + if (!content) return '' + content = content.replace(/\r?\n|\r/g, ' '); + + var extraSpace = /^`|^ .*?[^ ].* $|`$/.test(content) ? ' ' : ''; + var delimiter = '`'; + var matches = content.match(/`+/gm) || []; + while (matches.indexOf(delimiter) !== -1) delimiter = delimiter + '`'; + + return delimiter + extraSpace + content + extraSpace + delimiter + } + }; + + rules.image = { + filter: 'img', + + replacement: function (content, node) { + var alt = cleanAttribute(node.getAttribute('alt')); + var src = node.getAttribute('src') || ''; + var title = cleanAttribute(node.getAttribute('title')); + var titlePart = title ? ' "' + title + '"' : ''; + return src ? '![' + alt + ']' + '(' + src + titlePart + ')' : '' + } + }; + + function cleanAttribute (attribute) { + return attribute ? attribute.replace(/(\n+\s*)+/g, '\n') : '' + } + + /** + * Manages a collection of rules used to convert HTML to Markdown + */ + + function Rules (options) { + this.options = options; + this._keep = []; + this._remove = []; + + this.blankRule = { + replacement: options.blankReplacement + }; + + this.keepReplacement = options.keepReplacement; + + this.defaultRule = { + replacement: options.defaultReplacement + }; + + this.array = []; + for (var key in options.rules) this.array.push(options.rules[key]); + } + + Rules.prototype = { + add: function (key, rule) { + this.array.unshift(rule); + }, + + keep: function (filter) { + this._keep.unshift({ + filter: filter, + replacement: this.keepReplacement + }); + }, + + remove: function (filter) { + this._remove.unshift({ + filter: filter, + replacement: function () { + return '' + } + }); + }, + + forNode: function (node) { + if (node.isBlank) return this.blankRule + var rule; + + if ((rule = findRule(this.array, node, this.options))) return rule + if ((rule = findRule(this._keep, node, this.options))) return rule + if ((rule = findRule(this._remove, node, this.options))) return rule + + return this.defaultRule + }, + + forEach: function (fn) { + for (var i = 0; i < this.array.length; i++) fn(this.array[i], i); + } + }; + + function findRule (rules, node, options) { + for (var i = 0; i < rules.length; i++) { + var rule = rules[i]; + if (filterValue(rule, node, options)) return rule + } + return void 0 + } + + function filterValue (rule, node, options) { + var filter = rule.filter; + if (typeof filter === 'string') { + if (filter === node.nodeName.toLowerCase()) return true + } else if (Array.isArray(filter)) { + if (filter.indexOf(node.nodeName.toLowerCase()) > -1) return true + } else if (typeof filter === 'function') { + if (filter.call(rule, node, options)) return true + } else { + throw new TypeError('`filter` needs to be a string, array, or function') + } + } + + /** + * The collapseWhitespace function is adapted from collapse-whitespace + * by Luc Thevenard. + * + * The MIT License (MIT) + * + * Copyright (c) 2014 Luc Thevenard + * + * Permission is hereby granted, free of charge, to any person obtaining a copy + * of this software and associated documentation files (the "Software"), to deal + * in the Software without restriction, including without limitation the rights + * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell + * copies of the Software, and to permit persons to whom the Software is + * furnished to do so, subject to the following conditions: + * + * The above copyright notice and this permission notice shall be included in + * all copies or substantial portions of the Software. + * + * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR + * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, + * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE + * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER + * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, + * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN + * THE SOFTWARE. + */ + + /** + * collapseWhitespace(options) removes extraneous whitespace from an the given element. + * + * @param {Object} options + */ + function collapseWhitespace (options) { + var element = options.element; + var isBlock = options.isBlock; + var isVoid = options.isVoid; + var isPre = options.isPre || function (node) { + return node.nodeName === 'PRE' + }; + + if (!element.firstChild || isPre(element)) return + + var prevText = null; + var keepLeadingWs = false; + + var prev = null; + var node = next(prev, element, isPre); + + while (node !== element) { + if (node.nodeType === 3 || node.nodeType === 4) { // Node.TEXT_NODE or Node.CDATA_SECTION_NODE + var text = node.data.replace(/[ \r\n\t]+/g, ' '); + + if ((!prevText || / $/.test(prevText.data)) && + !keepLeadingWs && text[0] === ' ') { + text = text.substr(1); + } + + // `text` might be empty at this point. + if (!text) { + node = remove(node); + continue + } + + node.data = text; + + prevText = node; + } else if (node.nodeType === 1) { // Node.ELEMENT_NODE + if (isBlock(node) || node.nodeName === 'BR') { + if (prevText) { + prevText.data = prevText.data.replace(/ $/, ''); + } + + prevText = null; + keepLeadingWs = false; + } else if (isVoid(node) || isPre(node)) { + // Avoid trimming space around non-block, non-BR void elements and inline PRE. + prevText = null; + keepLeadingWs = true; + } else if (prevText) { + // Drop protection if set previously. + keepLeadingWs = false; + } + } else { + node = remove(node); + continue + } + + var nextNode = next(prev, node, isPre); + prev = node; + node = nextNode; + } + + if (prevText) { + prevText.data = prevText.data.replace(/ $/, ''); + if (!prevText.data) { + remove(prevText); + } + } + } + + /** + * remove(node) removes the given node from the DOM and returns the + * next node in the sequence. + * + * @param {Node} node + * @return {Node} node + */ + function remove (node) { + var next = node.nextSibling || node.parentNode; + + node.parentNode.removeChild(node); + + return next + } + + /** + * next(prev, current, isPre) returns the next node in the sequence, given the + * current and previous nodes. + * + * @param {Node} prev + * @param {Node} current + * @param {Function} isPre + * @return {Node} + */ + function next (prev, current, isPre) { + if ((prev && prev.parentNode === current) || isPre(current)) { + return current.nextSibling || current.parentNode + } + + return current.firstChild || current.nextSibling || current.parentNode + } + + /* + * Set up window for Node.js + */ + + var root = (typeof window !== 'undefined' ? window : {}); + + /* + * Parsing HTML strings + */ + + function canParseHTMLNatively () { + var Parser = root.DOMParser; + var canParse = false; + + // Adapted from https://gist.github.com/1129031 + // Firefox/Opera/IE throw errors on unsupported types + try { + // WebKit returns null on unsupported types + if (new Parser().parseFromString('', 'text/html')) { + canParse = true; + } + } catch (e) {} + + return canParse + } + + function createHTMLParser () { + var Parser = function () {}; + + { + if (shouldUseActiveX()) { + Parser.prototype.parseFromString = function (string) { + var doc = new window.ActiveXObject('htmlfile'); + doc.designMode = 'on'; // disable on-page scripts + doc.open(); + doc.write(string); + doc.close(); + return doc + }; + } else { + Parser.prototype.parseFromString = function (string) { + var doc = document.implementation.createHTMLDocument(''); + doc.open(); + doc.write(string); + doc.close(); + return doc + }; + } + } + return Parser + } + + function shouldUseActiveX () { + var useActiveX = false; + try { + document.implementation.createHTMLDocument('').open(); + } catch (e) { + if (window.ActiveXObject) useActiveX = true; + } + return useActiveX + } + + var HTMLParser = canParseHTMLNatively() ? root.DOMParser : createHTMLParser(); + + function RootNode (input, options) { + var root; + if (typeof input === 'string') { + var doc = htmlParser().parseFromString( + // DOM parsers arrange elements in the and . + // Wrapping in a custom element ensures elements are reliably arranged in + // a single element. + '' + input + '', + 'text/html' + ); + root = doc.getElementById('turndown-root'); + } else { + root = input.cloneNode(true); + } + collapseWhitespace({ + element: root, + isBlock: isBlock, + isVoid: isVoid, + isPre: options.preformattedCode ? isPreOrCode : null + }); + + return root + } + + var _htmlParser; + function htmlParser () { + _htmlParser = _htmlParser || new HTMLParser(); + return _htmlParser + } + + function isPreOrCode (node) { + return node.nodeName === 'PRE' || node.nodeName === 'CODE' + } + + function Node (node, options) { + node.isBlock = isBlock(node); + node.isCode = node.nodeName === 'CODE' || node.parentNode.isCode; + node.isBlank = isBlank(node); + node.flankingWhitespace = flankingWhitespace(node, options); + return node + } + + function isBlank (node) { + return ( + !isVoid(node) && + !isMeaningfulWhenBlank(node) && + /^\s*$/i.test(node.textContent) && + !hasVoid(node) && + !hasMeaningfulWhenBlank(node) + ) + } + + function flankingWhitespace (node, options) { + if (node.isBlock || (options.preformattedCode && node.isCode)) { + return { leading: '', trailing: '' } + } + + var edges = edgeWhitespace(node.textContent); + + // abandon leading ASCII WS if left-flanked by ASCII WS + if (edges.leadingAscii && isFlankedByWhitespace('left', node, options)) { + edges.leading = edges.leadingNonAscii; + } + + // abandon trailing ASCII WS if right-flanked by ASCII WS + if (edges.trailingAscii && isFlankedByWhitespace('right', node, options)) { + edges.trailing = edges.trailingNonAscii; + } + + return { leading: edges.leading, trailing: edges.trailing } + } + + function edgeWhitespace (string) { + var m = string.match(/^(([ \t\r\n]*)(\s*))[\s\S]*?((\s*?)([ \t\r\n]*))$/); + return { + leading: m[1], // whole string for whitespace-only strings + leadingAscii: m[2], + leadingNonAscii: m[3], + trailing: m[4], // empty for whitespace-only strings + trailingNonAscii: m[5], + trailingAscii: m[6] + } + } + + function isFlankedByWhitespace (side, node, options) { + var sibling; + var regExp; + var isFlanked; + + if (side === 'left') { + sibling = node.previousSibling; + regExp = / $/; + } else { + sibling = node.nextSibling; + regExp = /^ /; + } + + if (sibling) { + if (sibling.nodeType === 3) { + isFlanked = regExp.test(sibling.nodeValue); + } else if (options.preformattedCode && sibling.nodeName === 'CODE') { + isFlanked = false; + } else if (sibling.nodeType === 1 && !isBlock(sibling)) { + isFlanked = regExp.test(sibling.textContent); + } + } + return isFlanked + } + + var reduce = Array.prototype.reduce; + var escapes = [ + [/\\/g, '\\\\'], + [/\*/g, '\\*'], + [/^-/g, '\\-'], + [/^\+ /g, '\\+ '], + [/^(=+)/g, '\\$1'], + [/^(#{1,6}) /g, '\\$1 '], + [/`/g, '\\`'], + [/^~~~/g, '\\~~~'], + [/\[/g, '\\['], + [/\]/g, '\\]'], + [/^>/g, '\\>'], + [/_/g, '\\_'], + [/^(\d+)\. /g, '$1\\. '] + ]; + + function TurndownService (options) { + if (!(this instanceof TurndownService)) return new TurndownService(options) + + var defaults = { + rules: rules, + headingStyle: 'setext', + hr: '* * *', + bulletListMarker: '*', + codeBlockStyle: 'indented', + fence: '```', + emDelimiter: '_', + strongDelimiter: '**', + linkStyle: 'inlined', + linkReferenceStyle: 'full', + br: ' ', + preformattedCode: false, + blankReplacement: function (content, node) { + return node.isBlock ? '\n\n' : '' + }, + keepReplacement: function (content, node) { + return node.isBlock ? '\n\n' + node.outerHTML + '\n\n' : node.outerHTML + }, + defaultReplacement: function (content, node) { + return node.isBlock ? '\n\n' + content + '\n\n' : content + } + }; + this.options = extend({}, defaults, options); + this.rules = new Rules(this.options); + } + + TurndownService.prototype = { + /** + * The entry point for converting a string or DOM node to Markdown + * @public + * @param {String|HTMLElement} input The string or DOM node to convert + * @returns A Markdown representation of the input + * @type String + */ + + turndown: function (input) { + if (!canConvert(input)) { + throw new TypeError( + input + ' is not a string, or an element/document/fragment node.' + ) + } + + if (input === '') return '' + + var output = process.call(this, new RootNode(input, this.options)); + return postProcess.call(this, output) + }, + + /** + * Add one or more plugins + * @public + * @param {Function|Array} plugin The plugin or array of plugins to add + * @returns The Turndown instance for chaining + * @type Object + */ + + use: function (plugin) { + if (Array.isArray(plugin)) { + for (var i = 0; i < plugin.length; i++) this.use(plugin[i]); + } else if (typeof plugin === 'function') { + plugin(this); + } else { + throw new TypeError('plugin must be a Function or an Array of Functions') + } + return this + }, + + /** + * Adds a rule + * @public + * @param {String} key The unique key of the rule + * @param {Object} rule The rule + * @returns The Turndown instance for chaining + * @type Object + */ + + addRule: function (key, rule) { + this.rules.add(key, rule); + return this + }, + + /** + * Keep a node (as HTML) that matches the filter + * @public + * @param {String|Array|Function} filter The unique key of the rule + * @returns The Turndown instance for chaining + * @type Object + */ + + keep: function (filter) { + this.rules.keep(filter); + return this + }, + + /** + * Remove a node that matches the filter + * @public + * @param {String|Array|Function} filter The unique key of the rule + * @returns The Turndown instance for chaining + * @type Object + */ + + remove: function (filter) { + this.rules.remove(filter); + return this + }, + + /** + * Escapes Markdown syntax + * @public + * @param {String} string The string to escape + * @returns A string with Markdown syntax escaped + * @type String + */ + + escape: function (string) { + return escapes.reduce(function (accumulator, escape) { + return accumulator.replace(escape[0], escape[1]) + }, string) + } + }; + + /** + * Reduces a DOM node down to its Markdown string equivalent + * @private + * @param {HTMLElement} parentNode The node to convert + * @returns A Markdown representation of the node + * @type String + */ + + function process (parentNode) { + var self = this; + return reduce.call(parentNode.childNodes, function (output, node) { + node = new Node(node, self.options); + + var replacement = ''; + if (node.nodeType === 3) { + replacement = node.isCode ? node.nodeValue : self.escape(node.nodeValue); + } else if (node.nodeType === 1) { + replacement = replacementForNode.call(self, node); + } + + return join(output, replacement) + }, '') + } + + /** + * Appends strings as each rule requires and trims the output + * @private + * @param {String} output The conversion output + * @returns A trimmed version of the ouput + * @type String + */ + + function postProcess (output) { + var self = this; + this.rules.forEach(function (rule) { + if (typeof rule.append === 'function') { + output = join(output, rule.append(self.options)); + } + }); + + return output.replace(/^[\t\r\n]+/, '').replace(/[\t\r\n\s]+$/, '') + } + + /** + * Converts an element node to its Markdown equivalent + * @private + * @param {HTMLElement} node The node to convert + * @returns A Markdown representation of the node + * @type String + */ + + function replacementForNode (node) { + var rule = this.rules.forNode(node); + var content = process.call(this, node); + var whitespace = node.flankingWhitespace; + if (whitespace.leading || whitespace.trailing) content = content.trim(); + return ( + whitespace.leading + + rule.replacement(content, node, this.options) + + whitespace.trailing + ) + } + + /** + * Joins replacement to the current output with appropriate number of new lines + * @private + * @param {String} output The current conversion output + * @param {String} replacement The string to append to the output + * @returns Joined output + * @type String + */ + + function join (output, replacement) { + var s1 = trimTrailingNewlines(output); + var s2 = trimLeadingNewlines(replacement); + var nls = Math.max(output.length - s1.length, replacement.length - s2.length); + var separator = '\n\n'.substring(0, nls); + + return s1 + separator + s2 + } + + /** + * Determines whether an input can be converted + * @private + * @param {String|HTMLElement} input Describe this parameter + * @returns Describe what it returns + * @type String|Object|Array|Boolean|Number + */ + + function canConvert (input) { + return ( + input != null && ( + typeof input === 'string' || + (input.nodeType && ( + input.nodeType === 1 || input.nodeType === 9 || input.nodeType === 11 + )) + ) + ) + } + + return TurndownService; + +}))); diff --git a/www/pad/export.js b/www/pad/export.js index f1e3497c9..01ac27d7e 100644 --- a/www/pad/export.js +++ b/www/pad/export.js @@ -1,12 +1,24 @@ define([ 'jquery', '/common/common-util.js', + '/common/diffMarked.js', + '/common/hyperscript.js', '/bower_components/hyperjson/hyperjson.js', '/bower_components/nthen/index.js', -], function ($, Util, Hyperjson, nThen) { + '/lib/turndown.browser.umd.js' +], function ($, Util, DiffMd, h, Hyperjson, nThen, Turndown) { var module = { ext: '.html', // default - exts: ['.html', '.doc'] + exts: ['.html', '.md', '.doc'] + }; + + module.importMd = function (md, common) { + var html = DiffMd.render(md, true, false, true); + var div = h('div#cp-temp'); + DiffMd.apply(html, $(div), common); + var body = h('body'); + body.innerHTML = div.innerHTML; + return body; }; var exportMediaTags = function (inner, cb) { @@ -77,6 +89,15 @@ define([ }); return void cb(blob); } + if (ext === ".md") { + var md = Turndown({ + headingStyle: 'atx' + }).turndown(toExport); + var mdBlob = new Blob([md], { + type: 'text/markdown;charset=utf-8' + }); + return void cb(mdBlob); + } var html = module.getHTML(toExport); cb(new Blob([ html ], { type: "text/html;charset=utf-8" })); }); diff --git a/www/pad/inner.js b/www/pad/inner.js index 47ab5dbe3..304e9abe4 100644 --- a/www/pad/inner.js +++ b/www/pad/inner.js @@ -1115,7 +1115,7 @@ define([ framework._.sfCommon.isPadStored(function(err, val) { if (!val) { return; } - var b64images = $inner.find('img[src^="data:image"]:not(.cke_reset)'); + var b64images = $inner.find('img[src^="data:image"]:not(.cke_reset), img[src^="data:application/octet-stream"]:not(.cke_reset)'); if (b64images.length && framework._.sfCommon.isLoggedIn()) { var no = h('button.cp-corner-cancel', Messages.cancel); var yes = h('button.cp-corner-primary', Messages.ok); @@ -1169,7 +1169,14 @@ define([ }); cb($dom[0]); }; - framework.setFileImporter({ accept: 'text/html' }, function(content, f, cb) { + framework.setFileImporter({ accept: ['.md', 'text/html'] }, function(content, f, cb) { + if (!f) { return; } + if (/\.md$/.test(f.name)) { + var mdDom = Exporter.importMd(content, framework._.sfCommon); + return importMediaTags(mdDom, function(dom) { + cb(Hyperjson.fromDOM(dom)); + }); + } importMediaTags(domFromHTML(content).body, function(dom) { cb(Hyperjson.fromDOM(dom)); }); @@ -1320,6 +1327,7 @@ define([ $(waitFor()); }).nThen(function(waitFor) { Ckeditor.config.toolbarCanCollapse = true; + Ckeditor.config.language = Messages._getLanguage(); if (screen.height < 800) { Ckeditor.config.toolbarStartupExpanded = false; $('meta[name=viewport]').attr('content', diff --git a/www/poll/inner.js b/www/poll/inner.js index 6d4e0590e..53faea7e9 100644 --- a/www/poll/inner.js +++ b/www/poll/inner.js @@ -955,7 +955,7 @@ define([ var fileHost = privateData.fileHost || privateData.origin; var src = fileHost + Hash.getBlobPathFromHex(secret.channel); var key = Hash.encodeBase64(secret.keys.cryptKey); - var mt = ''; + var mt = UI.mediaTag(src, key).outerHTML; APP.editor.replaceSelection(mt); } }; @@ -1235,7 +1235,7 @@ define([ common.openFilePicker(pickerCfg, function (data) { if (data.type === 'file' && APP.editor) { common.setPadAttribute('atime', +new Date(), null, data.href); - var mt = ''; + var mt = UI.mediaTag(data.src, data.key).outerHTML; APP.editor.replaceSelection(mt); return; } diff --git a/www/register/main.js b/www/register/main.js index 4c63e4482..4d9274253 100644 --- a/www/register/main.js +++ b/www/register/main.js @@ -47,7 +47,11 @@ define([ var I_REALLY_WANT_TO_USE_MY_EMAIL_FOR_MY_USERNAME = false; var registerClick = function () { - var uname = $uname.val(); + var uname = $uname.val().trim(); + // trim whitespace surrounding the username since it is otherwise included in key derivation + // most people won't realize that its presence is significant + $uname.val(uname); + var passwd = $passwd.val(); var confirmPassword = $confirm.val(); diff --git a/www/slide/app-slide.less b/www/slide/app-slide.less index 7e518a1c7..ce12b1de3 100644 --- a/www/slide/app-slide.less +++ b/www/slide/app-slide.less @@ -356,6 +356,7 @@ } .markdown_main(); + .markdown_cryptpad(); .markdown_preformatted-code; .markdown_gfm-table(); diff --git a/www/slide/inner.js b/www/slide/inner.js index 3eb1435ed..e40bd6d79 100644 --- a/www/slide/inner.js +++ b/www/slide/inner.js @@ -556,7 +556,7 @@ define([ var fileHost = privateData.fileHost || privateData.origin; var src = fileHost + Hash.getBlobPathFromHex(secret.channel); var key = Hash.encodeBase64(secret.keys.cryptKey); - var mt = ''; + var mt = UI.mediaTag(src, key).outerHTML; editor.replaceSelection(mt); } }; diff --git a/www/support/inner.js b/www/support/inner.js index f466e7693..6d9e5eae8 100644 --- a/www/support/inner.js +++ b/www/support/inner.js @@ -272,19 +272,27 @@ define([ APP.$rightside = $('
    ', {id: 'cp-sidebarlayout-rightside'}).appendTo(APP.$container); var sFrameChan = common.getSframeChannel(); sFrameChan.onReady(waitFor()); + }).nThen(function (waitFor) { + metadataMgr = common.getMetadataMgr(); + privateData = metadataMgr.getPrivateData(); common.getPinUsage(null, waitFor(function (err, data) { if (err) { return void console.error(err); } APP.pinUsage = data; })); + APP.teamsUsage = {}; + Object.keys(privateData.teams).forEach(function (teamId) { + common.getPinUsage(teamId, waitFor(function (err, data) { + if (err) { return void console.error(err); } + APP.teamsUsage[teamId] = data; + })); + }); }).nThen(function (/*waitFor*/) { createToolbar(); - metadataMgr = common.getMetadataMgr(); - privateData = metadataMgr.getPrivateData(); common.setTabTitle(Messages.supportPage); APP.origin = privateData.origin; APP.readOnly = privateData.readOnly; - APP.support = Support.create(common, false, APP.pinUsage); + APP.support = Support.create(common, false, APP.pinUsage, APP.teamsUsage); // Content var $rightside = APP.$rightside; diff --git a/www/support/ui.js b/www/support/ui.js index 679776c59..90fd43fa5 100644 --- a/www/support/ui.js +++ b/www/support/ui.js @@ -31,9 +31,7 @@ define([ if (typeof(ctx.pinUsage) === 'object') { // pass pin.usage, pin.limit, and pin.plan if supplied - Object.keys(ctx.pinUsage).forEach(function (k) { - data.sender[k] = ctx.pinUsage[k]; - }); + data.sender.quota = ctx.pinUsage; } data.id = id; @@ -45,11 +43,14 @@ define([ data.sender.blockLocation = privateData.blockLocation || ''; data.sender.teams = Object.keys(teams).map(function (key) { var team = teams[key]; - if (!teams) { return; } + if (!team) { return; } var ret = {}; - ['edPublic', 'owner', 'viewer', 'hasSecondaryKey', 'validKeys'].forEach(function (k) { + ['channel', 'roster', 'numberPads', 'numberSf', 'edPublic', 'curvePublic', 'owner', 'viewer', 'hasSecondaryKey', 'validKeys'].forEach(function (k) { ret[k] = team[k]; }); + if (ctx.teamsUsage && ctx.teamsUsage[key]) { + ret.quota = ctx.teamsUsage[key]; + } return ret; }).filter(Boolean); @@ -430,12 +431,13 @@ define([ ]); }; - var create = function (common, isAdmin, pinUsage) { + var create = function (common, isAdmin, pinUsage, teamsUsage) { var ui = {}; var ctx = { common: common, isAdmin: isAdmin, pinUsage: pinUsage || false, + teamsUsage: teamsUsage || false, adminKeys: Array.isArray(ApiConfig.adminKeys)? ApiConfig.adminKeys.slice(): [], }; diff --git a/www/teams/inner.js b/www/teams/inner.js index 4b0532628..373a27998 100644 --- a/www/teams/inner.js +++ b/www/teams/inner.js @@ -768,7 +768,7 @@ define([ $(demote).hide(); describeUser(common, data.curvePublic, { role: role - }, promote); + }, demote); }; if (isMe) { return void UI.confirm(Messages.team_demoteMeConfirm, function (yes) { @@ -901,22 +901,23 @@ define([ $header.append(invite); } - if (me && (me.role !== 'OWNER')) { - var leave = h('button.cp-online.btn.btn-danger', Messages.team_leaveButton); - $(leave).click(function () { - UI.confirm(Messages.team_leaveConfirm, function (yes) { - if (!yes) { return; } - APP.module.execCommand('LEAVE_TEAM', { - teamId: APP.team - }, function (obj) { - if (obj && obj.error) { - return void UI.warn(Messages.error); - } - }); + var leave = h('button.cp-online.btn.btn-danger', Messages.team_leaveButton); + $(leave).click(function () { + if (me && me.role === 'OWNER') { + return void UI.alert(Messages.team_leaveOwner); + } + UI.confirm(Messages.team_leaveConfirm, function (yes) { + if (!yes) { return; } + APP.module.execCommand('LEAVE_TEAM', { + teamId: APP.team + }, function (obj) { + if (obj && obj.error) { + return void UI.warn(Messages.error); + } }); }); - $header.append(leave); - } + }); + $header.append(leave); var table = h('button.btn.btn-primary', Messages.teams_table); $(table).click(function (e) {