e6a4c68863
always delete the throttled cleanup functions once they have run
4 years ago
4495d99e49
use the standard method of checking whether metadata is valid
4 years ago
6bc103be5f
delay cache eviction each time the metadata is requested
4 years ago
d4f94f3091
briefly cache server metadata in-memory to avoid repeated reads
4 years ago
34284fcc0c
Fix issues with deprecated cache
4 years ago
820f2a0054
Merge branch 'soon' into staging
4 years ago
93d56e5e3d
profile worker commands by total running time in seconds
4 years ago
93c807524f
update offset debugging info
4 years ago
5bbde5fec5
Merge branch 'soon' into cacheRT
4 years ago
d4e6d86961
enforce asynchrony in new database offset methods
4 years ago
466072d03b
read files starting from the oldest known point of relevance when computing indices
4 years ago
93ddf51564
Merge branch 'download' into cacheRT
4 years ago
cd77b5ffc8
Merge branch 'soon' into staging
4 years ago
faa84bcbd7
describe arguments for 'completeUpload' RPC
4 years ago
bd4b44476d
Archive and restore UI in the admin panel
4 years ago
e185008846
complete uploads in child processes
...
also fix a nasty race condition for unowned file uploads
4 years ago
b155f7b291
increase child-process timeout and improve error handling
4 years ago
ead97d8f41
Remove FAQ and references to it
4 years ago
5946b10d0b
No page reload when the cache is corrupted onReady
4 years ago
1f1b40b83d
move some implictly global state to env.js
4 years ago
aa547a7b76
align nodejs http headers with example nginx
4 years ago
5cf8ca70e6
don't pin falsey document ids
4 years ago
c9a3cad78b
Fix various issues with restricted pads
4 years ago
d16cc2472d
First prototype of an offline cache
4 years ago
66d434b583
Remove about page from defaults and add XXX for future pages
4 years ago
034472d3dd
move some implictly global state to env.js
4 years ago
0bc2433e02
Merge branch 'accessBug' into staging
4 years ago
d95b0954f9
Merge branch 'fix-printing' into staging
4 years ago
a2b79d84b8
align nodejs http headers with example nginx
4 years ago
f7bd3bdc23
don't pin falsey document ids
4 years ago
69c26fe8c7
Fix various issues with restricted pads
4 years ago
d72e053560
make a note to improve stream timeout error handling
4 years ago
67430de7ff
Make efforts to avoid closing streams mid-read
...
1. Close streams when you're done with them
2. Close streams if they seem to have been abandoned
4 years ago
fbfb25bf29
lint compliance
4 years ago
100b417646
guard against several serverside typeErrors
...
and warn in cases where they would have occurred
4 years ago
e8428a2a73
prevent a case of multiple callbacks
4 years ago
fbc9edd795
rename latest admin commands and implement metadata getter/remover
4 years ago
488ec93ece
allow expert admins to get and clear cached channel indices
4 years ago
faa7ebf399
Fix some bugs with integrated eviction
...
1. implement SET_LAST_EVICTION as an admin command, not a decree
2. expect a return value from Env.evictInactive and expose it via Env.evictionReport
4 years ago
ac322c8e82
make the eviction script call back with accumulated statistics
4 years ago
a29c5641b8
introduce some artificial delays in the eviction script
...
...and reduce the margin of error in its bloom filters
4 years ago
90f046f896
prevent long-running worker tasks from timing out
4 years ago
6e57366b7f
address a file descriptor leak
4 years ago
a2c0d2165b
implement a few more admin decrees
4 years ago
abd84665ae
lint compliance
4 years ago
6ec5171518
add support for changing a few more Env parameters at runtime
4 years ago
adb988058d
stop logging dev/fresh mode
4 years ago
f2ec9cbe33
load premium and customLimits to avoid evicting them even if inactive
4 years ago
45b063e378
suppress irrelevant errors from Decrees.load
4 years ago
854e4c06ff
report lastEviction in an admin INSTANCE_STATUS
4 years ago
471a9a33ac
take note of a minor error with pin RPCs
4 years ago
f86409e1ba
take note of which Environment variables should be made mutable via decree
4 years ago
6d13a785c6
downgrade a non-critical XXX to a FIXME
4 years ago
46ebd7b40b
remove unsupported storage configuration parameters
4 years ago
369c92c01d
initialize Env from server and deduplicate several attributes
4 years ago
b5d3a10dc2
guard against possible typeErrors from unvalidated config input
4 years ago
964aa2bb79
validate custom limits provided via config at launch
4 years ago
f4f803ccd9
reuse Keys.canonicalize to accept any pubkey format
4 years ago
914139dbfa
lint compliance
4 years ago
51e1b9b043
Fix custom limit not applied when using a raw edPublic
4 years ago
145386e944
suppress an inconsequential error
4 years ago
aa35e092f7
update an inaccurate comment
4 years ago
928b7e3583
implement GET_LIMITS admin rpc
...
...and label stored usage limits according to their source
4 years ago
5029281a53
implement RM_QUOTA decree
4 years ago
af47b2cab1
implement ADD_QUOTA admin decree
4 years ago
998f2267c4
ensure the path to the decree file exists before attempting to write to it
4 years ago
c2918ac888
fix inverted conditional
4 years ago
b62d4387a6
drop support for SET_DEFAULT_STORAGE_LIMIT
...
it's redundant now that we support persistent admin decrees
and it was never exposed via the UI anyway
4 years ago
0c5db31fbc
refactor quota logic for external usage
4 years ago
bd6234c5bc
lint compliance and some notes
4 years ago
1781ee2585
implement 'UPDATE_DEFAULT_STORAGE' as a decree
4 years ago
7aa7d5978f
implement persistence of config changes from admin panel
4 years ago
fcfcb3dca6
fix history trim for channels with access lists
4 years ago
d509fd8b8f
downgrade severity of several XXX notes
4 years ago
1ba080739b
Merge branch 'X' into staging
4 years ago
cfcfe2f65f
enable APIs for XLSX export in firefox
4 years ago
e698241ee9
Merge branch 'historyOO' into staging
4 years ago
04d1bc9cd7
archive inactive data in one of the server's workers
4 years ago
6ada5fd751
always consider premium accounts to be active
4 years ago
5358fab615
more WIP inactive account eviction
4 years ago
a9b95865ba
Merge branch 'staging' into inactive-accounts
4 years ago
d9219f4ecc
add a note for later
4 years ago
5749b6fd7b
add an admin RPC to archive documents
4 years ago
d794e0b48f
include inactive accounts in the archival script
4 years ago
25606b92bf
lint compliance
4 years ago
4f147d4fd2
Add support for onlyoffice history
4 years ago
b31707098a
Improve getHistoryRange to support 'toHash'
4 years ago
95778cc7c9
respond with ENOENT instead of E_NO_OWNERS when archiving a file that does not exist
4 years ago
b04133e6c3
remove some dead code
4 years ago
a21c3372c0
avoid a potential type error
4 years ago
04af753a81
lint compliance
4 years ago
03be102ce4
deduplicate some code by reusing the client definition on the server
4 years ago
ebcc9a069b
add server-side support for the new format of public signing keys
4 years ago
48deced859
fix serverside index corruption
5 years ago
5b2929a6fc
suppress some useless info from some commonly logged errors
5 years ago
9f1f01f3b4
disable worker task queueing and address a probable memory leak
5 years ago
b0af6b5488
fix an incorrect condition when checking for available server workers
...
Also, log when the RPC queue has a backlog and when it is drained
5 years ago
4037270a23
add a FIXME note for a sketchy implementation in history keeper
5 years ago
02347d134a
queue signature validation per-channel to ensure correct ordering
5 years ago
9975c27e84
increase timeout from one minute to three and decrease the number of parallel jobs per worker
5 years ago
488f876ad6
lint compliance
5 years ago
d35b015c3f
log some info on successful file uploads
5 years ago
37d3995ac1
unify format of console output with stored logs
5 years ago
8a5d012edf
add special error handling for a few special serverside errors
5 years ago
8bee95d13d
improve error reporting in GET_HISTORY
5 years ago
e05891184e
fix regression in maxWorkers
5 years ago
b56c73be6e
delegate more work from getOlderHistory to the worker
5 years ago
fd169ff39c
close streams after 2 minutes instead of 5
...
Hopefully this helps to even out our memory and CPU usage.
There's also some improved labels so that IF streams time out
we will know the exact origin of the offending call.
5 years ago
bac8ef3c6e
lint compliance
5 years ago
4e57e390da
write tasks inside of workers
5 years ago
ba6faca02e
make the number of workers configurable
5 years ago
0465f31a45
add a FIXME note
5 years ago
04ab7f538a
recover from worker faults and unify worker types
5 years ago
e0a6852b79
correctly serialize an error log
5 years ago
9ed82640bf
Merge branch 'worker-queue' into staging
5 years ago
e8b1fcf710
solve some cases where crashing workers could result in an invalid state for the queue
5 years ago
9ed26cfeb0
lint compliance
5 years ago
dc0460780f
Merge branch 'soon' into staging
5 years ago
74567e0cf4
return process memory usage in GET CACHE STATS RPC
5 years ago
e3f42cbc20
Allow config.defaultStorageLimit to be zero bytes
5 years ago
89262cd29e
resolve silly conflict and merge staging
5 years ago
e527867e2e
queue blob and channel deletions per-user
5 years ago
bd3e18d1a8
only cache a user's pinned channels if there are no errors when loading them
5 years ago
bef18a9320
hotfix
5 years ago
7fac997e93
increase some file storage timeouts related to streams
5 years ago
95965c1dee
keep a parallel implementation of the pin loader to validate the new one
5 years ago
a57bf94058
remove XXX note
5 years ago
c53675c9d5
WIP worker rpc call queue
5 years ago
d802173325
remove an invalid line
5 years ago
2b3f6e3464
Merge branch 'soon' into staging
5 years ago
adfd1eaaa2
avoid a silly typeError in the event of an error
5 years ago
32406a5995
Merge branch 'master' into staging
5 years ago
834c96a4fc
update comments and remove some notes
5 years ago
ed5d97f899
WIP history-keeper fixes
5 years ago
eddbe80eef
Merge branch 'soon' into staging
5 years ago
3d8e78a268
fix overly broad config interpretation for daily check
5 years ago
e6ec891d9a
improve admin rpc for diagnosing memory used by the cache
5 years ago
77961e3954
trim lookup tables for channels without checkpoints
5 years ago
db9eaa8d4f
Merge branch 'soon' into staging
5 years ago
6d0dee979a
allow admins to inspect index/metadata caches
5 years ago
cbd3547814
remove log statements
5 years ago
9058a59555
reassign db tasks if the responsible worker fails
5 years ago
172823c954
lint compliance
5 years ago
b5649707d1
export 'tryParse' command
5 years ago
64b0879984
improve logging for parse errors
5 years ago
3f86b6141e
rename worker processes
5 years ago
d8a88cb4ca
run expiration tasks in a worker instead of the main process
5 years ago
b0179eaad9
drop XXX note
5 years ago
9dbd32758a
improve worker logging and move blob deletion to worker processes as well
5 years ago
5f69fc18d0
suppress some noisy errors
5 years ago
ansuz