Commit Graph

221 Commits (eddbe80eeff80bc0892a071642d7c644b9b80a21)

Author SHA1 Message Date
ansuz f8ad649b45 [style] bail out early to avoid nesting 2020-03-18 10:30:42 -04:00
ansuz 019e5e708b wrap workers in a function scope and add a validateMessage method to HK's Env 2020-03-17 17:23:35 -04:00
ansuz 5467e1ffac replace ad-hoc response handler with Util.response 2020-03-17 16:52:41 -04:00
ansuz 9e85a1411e abstract the logic around worker choice out of message handling 2020-03-17 16:06:01 -04:00
ansuz 631ea54b49 lint compliance 2020-03-17 15:40:06 -04:00
ansuz 67dd4a608c Merge remote-tracking branch 'origin/scaling' into scaling2 2020-03-17 15:36:58 -04:00
ansuz b2fcde87d8 implement an admin command to update the default storage limit without a restart 2020-03-17 10:11:52 -04:00
yflory 923616aef0 Use each process in order instead of using a random one 2020-03-17 15:10:23 +01:00
ansuz f345998956 fix undefined reference 2020-03-17 10:10:20 -04:00
yflory fb0eb1b20c Use more subprocesses 2020-03-17 14:18:41 +01:00
yflory 0d636dabc9 Check signature for history keeper in a different process 2020-03-17 13:29:53 +01:00
ansuz 30f17040ac close streams when possible, simplify some code, enforce asynchrony 2020-03-16 16:13:38 -04:00
ansuz 2a9780abcf lint compliance 2020-03-16 12:29:26 -04:00
yflory 578042154e Better reconnect after invalid lastKnownHash 2020-03-13 17:39:16 +01:00
ansuz 2d47e7e2cf implement an admin rpc to fetch ulimit data 2020-03-12 12:09:44 -04:00
ansuz 81c7416095 lint compliance 2020-03-12 11:40:46 -04:00
ansuz f644c24362 add an admin-rpc to count the number of open files 2020-03-12 11:35:16 -04:00
ansuz 1230057970 disable IS_PAD_PINNED 2020-03-12 11:31:08 -04:00
ansuz e1a0daac9c possible server fixes 2020-03-12 11:23:43 -04:00
ansuz bc034d95a3 enforce asynchrony in 'gethistoryOffset' 2020-03-12 11:13:23 -04:00
ansuz d4038344ba stop iterating over tasks if you can't read the root directory 2020-03-11 16:34:00 -04:00
ansuz 4e8ebac3c1 guard against the possible non-existence of metadata for a channel 2020-03-09 17:42:10 -04:00
ansuz b32f3e34af fix local worker debugging on firefox 2020-03-06 11:43:34 -05:00
ansuz 6d8cdca7a3 add 'rejected' field to metadata when getMetadata requests are blocked 2020-03-05 12:09:18 -05:00
ansuz 9d8bb43d03 finally untangle metadata and index caches 2020-03-04 12:56:33 -05:00
ansuz 32d769447a begin standardizing our method of streaming lines from files 2020-03-04 11:38:14 -05:00
ansuz 35eca2c5d2 pull file streaming out into its own file, leave a few notes 2020-03-04 11:38:14 -05:00
ansuz ce6cf6f727 revert some changes that couldn't have possibly worked 2020-03-04 09:59:17 -05:00
ansuz b126e4456b simplify quota logic 2020-03-03 16:55:40 -05:00
ansuz 53ed247bc2 expose maxUploadSize and premiumUploadSize via /api/config 2020-03-03 16:47:44 -05:00
ansuz 170aa6d47e clean up a few tasks related to allow lists 2020-03-03 15:52:49 -05:00
ansuz 2f00ff3278 simplify check-in deactivation 2020-02-28 14:24:53 -05:00
ansuz 03ff9bd0d7 start deprecating usage of myDomain 2020-02-28 11:45:41 -05:00
ansuz de6b9e2228 apply some defaults when loading the config 2020-02-28 10:50:30 -05:00
ansuz 94d8e7f2c2 provide a consistent type for 'customLimits' 2020-02-28 10:49:20 -05:00
ansuz 3cf09924ae fix default CSP headers 2020-02-28 10:46:44 -05:00
ansuz 8d509fd6d5 Merge branch 'staging' into small-config 2020-02-27 15:25:39 -05:00
ansuz c26560e3f9 commit stashed admin-rpc refactor 2020-02-27 15:00:31 -05:00
ansuz 02fc343727 Merge branch 'staging' into small-config 2020-02-27 13:54:15 -05:00
ansuz 3df47a1415 continued WIP config reorganization 2020-02-27 13:24:19 -05:00
ansuz 294a444603 WIP removing defaults from the example config file 2020-02-27 13:09:12 -05:00
ansuz 59ad80d7f1 support larger upload sizes for users with premium accounts 2020-02-26 13:09:11 -05:00
ansuz 3f2ebb773b merge master back to staging 2020-02-25 11:20:20 -05:00
ansuz 542150b775 merge server components from staging 2020-02-25 10:09:35 -05:00
ansuz f951951077 merge communities-allow-list and lint compliance 2020-02-25 09:48:41 -05:00
ansuz f0e5fc7614 merge some missing code from communities-allow-list 2020-02-25 09:09:16 -05:00
ansuz ad08fe0c08 silence a few more routine websocket errors 2020-02-25 08:01:25 -05:00
ansuz 021a6b361f remove a bogus XXX note 2020-02-21 14:33:28 -05:00
ansuz 033bd361f9 add backwards compatibility with older servers 2020-02-21 13:58:21 -05:00
ansuz cac114bb52 merge staging into pending allow-list work 2020-02-21 13:52:10 -05:00
ansuz 386add7c58 fix an invalid use of dropChannel 2020-02-21 13:46:11 -05:00
ansuz 33c12c27a1 ignore successive metadata errors with the same message 2020-02-21 13:38:56 -05:00
ansuz 7ee8778210 kick unauthorized users from restricted channels
and update dependencies
2020-02-21 13:05:24 -05:00
ansuz fe0e02a421 restrict access to metadata 2020-02-21 10:31:10 -05:00
ansuz 791aad53f2 WIP allow list changes 2020-02-21 08:18:05 -05:00
ansuz 597f417ad6 WIP block history requests that bypass JOIN restriction (where appropriate) 2020-02-19 15:32:15 -05:00
ansuz 79bc8830ef disable access restriction if all owners are removed 2020-02-19 14:39:28 -05:00
ansuz f579c9b059 semi-functional allow-list implementation in historyKeeper 2020-02-19 14:22:12 -05:00
ansuz 0edcdcb1b8 oups! fix a missing callback that's been here for a while... 2020-02-18 16:15:05 -05:00
ansuz 27e57e7af0 oups! fix a missing callback that's been here for a while... 2020-02-18 16:07:14 -05:00
ansuz eac3e4cbcd sketch out some historyKeeper functionality related to access lists 2020-02-18 16:05:12 -05:00
ansuz 156c37942d implement new metadata commands related to allow lists and mailboxes 2020-02-18 14:16:07 -05:00
ansuz a280cc85d2 Merge branch 'staging' into communities-allow-list 2020-02-17 12:04:41 -05:00
ansuz 626a031115 resolve conflict with master/staging 2020-02-17 12:04:01 -05:00
ansuz ff73e96cb8 reimplement the trim history fix from staging 2020-02-17 12:02:49 -05:00
ansuz a4be6185de merge staging and do a little lint compliance 2020-02-17 11:54:27 -05:00
ansuz f478ae725d Merge branch 'staging' into communities-allow-list 2020-02-17 11:48:56 -05:00
ansuz 79f1280cf3 Merge branch 'soon' into staging 2020-02-17 11:42:08 -05:00
ansuz b56367414b don't overwrite cached indices when new users join a channel 2020-02-17 11:31:00 -05:00
ansuz 253ea0d336 Merge branch 'staging' into communities-allow-list 2020-02-17 10:38:40 -05:00
ansuz e8949168ec lint compliance 2020-02-17 10:30:44 -05:00
ansuz 5dff6535ed add a simple guard against unparsed messages when trimming history 2020-02-17 10:19:51 -05:00
ansuz 8694c17023 sketch out metadata commands for allow lists 2020-02-17 09:26:36 -05:00
ansuz f86196e40a implement shared environment between historyKeeper and RPC 2020-02-17 09:01:10 -05:00
ansuz 3f606d8c75 remove some duplicated code 2020-02-14 16:45:53 -05:00
ansuz 725d10fc60 nest storage directory inside './lib' 2020-02-14 16:29:30 -05:00
ansuz 38c1700173 Respond to pinning RPCs as soon as possible
(instead of waiting until you've read an unbounded number
of pin logs while queries back up in memory)

Also replace instances of 'publicKey' with 'safeKey' or 'unsafeKey'
to clearly and correctly indicate their format.
2020-02-14 13:46:40 -05:00
ansuz cded52f83f replicate existing pinned.load API correctly 2020-02-14 11:45:51 -05:00
ansuz 65f88617cf add support for an optional handler for each pin log
great for analyzing which files are most in need of optimization
2020-02-13 18:16:32 -05:00
ansuz 3dc789cbca replace pinned.js with low-profile streaming implementation 2020-02-13 17:46:10 -05:00
ansuz 609eddc9e1 reimplement the the Pinned.load API
* recurse into directories on demand
* optimized for lower memory consumption
* less parallelism than before
2020-02-13 17:41:36 -05:00
ansuz 2345323f0d lint compliance 2020-02-13 17:40:55 -05:00
ansuz 2d6626234b yet another async scheduler, this time supporting flexible runtime control flow based on declarative priority levels 2020-02-13 16:44:11 -05:00
yflory 0f697ac865 Merge branch 'staging' into communities-trim 2020-02-11 11:10:12 +01:00
ansuz 1231cacef1 remove XXX 2020-02-10 11:21:04 -05:00
ansuz 57be324850 remove some XXX notes 2020-02-10 10:01:02 -05:00
ansuz 68a77885d4 suppress some routine server logs 2020-02-10 09:03:16 -05:00
ansuz 7f22da0f57 fix use of incorrect key format in admin-rpc 2020-02-10 08:07:29 -05:00
ansuz 7000be0e70 clean up historykeeper 2020-02-07 18:58:57 -05:00
ansuz 802034616c centralize historykeeper-rpc interaction in rpc methods 2020-02-07 12:53:12 -05:00
yflory fd095644a2 Merge branch 'staging' into communities-trim 2020-02-06 11:55:07 +01:00
yflory 64c0f3a90d Add missing RPC command 2020-02-06 11:54:46 +01:00
yflory 9a857ea058 Merge branch 'staging' into communities-trim 2020-02-06 10:55:12 +01:00
ansuz b4172676f3 lint compliance 2020-02-05 18:12:09 -05:00
ansuz f17d14fd99 simpler limit updates 2020-02-05 18:01:48 -05:00
ansuz bde17a62a1 continue refactoring rpc 2020-02-05 17:31:44 -05:00
ansuz fa525817ba fix broken removePins RPC 2020-02-05 13:05:06 -05:00
ansuz a00a9fd018 don't hang when reading unparseable lines from history 2020-02-05 13:05:05 -05:00
yflory 1d56c08700 Merge branch 'staging' into communities-trim 2020-02-05 18:23:10 +01:00
ansuz 3741cbcbc0 rewrite trimmed history WITH newlines 2020-02-05 12:22:53 -05:00