d4e6d86961
enforce asynchrony in new database offset methods
4 years ago
466072d03b
read files starting from the oldest known point of relevance when computing indices
4 years ago
e185008846
complete uploads in child processes
...
also fix a nasty race condition for unowned file uploads
4 years ago
d72e053560
make a note to improve stream timeout error handling
4 years ago
67430de7ff
Make efforts to avoid closing streams mid-read
...
1. Close streams when you're done with them
2. Close streams if they seem to have been abandoned
4 years ago
e8428a2a73
prevent a case of multiple callbacks
4 years ago
6e57366b7f
address a file descriptor leak
4 years ago
46ebd7b40b
remove unsupported storage configuration parameters
4 years ago
5358fab615
more WIP inactive account eviction
4 years ago
d794e0b48f
include inactive accounts in the archival script
4 years ago
b04133e6c3
remove some dead code
4 years ago
fd169ff39c
close streams after 2 minutes instead of 5
...
Hopefully this helps to even out our memory and CPU usage.
There's also some improved labels so that IF streams time out
we will know the exact origin of the offending call.
5 years ago
7fac997e93
increase some file storage timeouts related to streams
5 years ago
d802173325
remove an invalid line
5 years ago
adfd1eaaa2
avoid a silly typeError in the event of an error
5 years ago
bc13a21796
give fileStreams a little bit more time before closing them
5 years ago
c2fcba31e3
remove an XXX
5 years ago
e128683ffb
call back with errors instead of throwing within the Storage module
5 years ago
dc0b26da0b
Merge branch 'soon' into staging
5 years ago
ed63d98ab0
Merge branch 'scaling-index' into soon
5 years ago
590e249500
resolve merge conflict
5 years ago
31c7cecaf9
add a few checks for safety
5 years ago
d386e223e4
simplify open/close of writeStreams
5 years ago
32cd0f3c4d
increase timeout value from 15s to 45. ought to be enough for anybody
5 years ago
05a4e86cdb
close streams whenever we finish using them. time out if necessary
5 years ago
4522ffa18a
compute indexes in child processes
5 years ago
f345998956
fix undefined reference
5 years ago
30f17040ac
close streams when possible, simplify some code, enforce asynchrony
5 years ago
e1a0daac9c
possible server fixes
5 years ago
d4038344ba
stop iterating over tasks if you can't read the root directory
5 years ago
32d769447a
begin standardizing our method of streaming lines from files
5 years ago
35eca2c5d2
pull file streaming out into its own file, leave a few notes
5 years ago
ff73e96cb8
reimplement the trim history fix from staging
5 years ago
725d10fc60
nest storage directory inside './lib'
5 years ago
ansuz