7bdabb5cbc
archive blocks before overwriting them
2021-07-06 15:42:37 +05:30
f7f2146fa5
miscellaneous cleanup and notes
2021-06-30 19:09:53 +05:30
ba1a7b37e1
separate validation and storage methods for blocks
2021-06-28 14:39:26 +05:30
caece0123e
archive pin logs instead of removing them outright
2021-06-25 20:22:16 +05:30
729d51fb9b
update block storage APIs
2021-04-29 13:32:19 +05:30
7971a6d267
Major broadcast refactoring
2021-03-16 17:00:23 +01:00
de90949e3e
suppress ENOENT on metadata when restoring archived files
2020-12-11 20:12:36 +05:30
5e7eb4cbf8
Merge branch 'soon' into staging
2020-12-04 13:39:47 +00:00
d4e6d86961
enforce asynchrony in new database offset methods
2020-11-27 08:21:51 +05:30
466072d03b
read files starting from the oldest known point of relevance when computing indices
2020-11-26 12:15:02 +05:30
cd77b5ffc8
Merge branch 'soon' into staging
2020-11-19 13:18:08 +05:30
bd4b44476d
Archive and restore UI in the admin panel
2020-11-18 16:49:40 +01:00
e185008846
complete uploads in child processes
...
also fix a nasty race condition for unowned file uploads
2020-11-18 18:26:21 +05:30
d72e053560
make a note to improve stream timeout error handling
2020-10-22 11:24:58 +05:30
67430de7ff
Make efforts to avoid closing streams mid-read
...
1. Close streams when you're done with them
2. Close streams if they seem to have been abandoned
2020-10-22 11:17:03 +05:30
e8428a2a73
prevent a case of multiple callbacks
2020-10-21 18:13:10 +05:30
6e57366b7f
address a file descriptor leak
2020-10-13 13:14:39 +05:30
46ebd7b40b
remove unsupported storage configuration parameters
2020-10-12 17:40:21 +05:30
5358fab615
more WIP inactive account eviction
2020-09-29 18:04:51 +05:30
d794e0b48f
include inactive accounts in the archival script
2020-09-24 19:02:22 +05:30
b04133e6c3
remove some dead code
2020-07-23 09:28:00 -04:00
fd169ff39c
close streams after 2 minutes instead of 5
...
Hopefully this helps to even out our memory and CPU usage.
There's also some improved labels so that IF streams time out
we will know the exact origin of the offending call.
2020-04-17 15:05:48 -04:00
7fac997e93
increase some file storage timeouts related to streams
2020-04-08 09:38:48 -04:00
d802173325
remove an invalid line
2020-04-06 10:18:11 -04:00
adfd1eaaa2
avoid a silly typeError in the event of an error
2020-04-02 22:57:57 -04:00
bc13a21796
give fileStreams a little bit more time before closing them
2020-03-25 11:50:45 -04:00
c2fcba31e3
remove an XXX
2020-03-24 15:02:46 -04:00
e128683ffb
call back with errors instead of throwing within the Storage module
2020-03-24 13:31:03 -04:00
dc0b26da0b
Merge branch 'soon' into staging
2020-03-23 12:33:17 +01:00
ed63d98ab0
Merge branch 'scaling-index' into soon
2020-03-20 18:57:51 -04:00
590e249500
resolve merge conflict
2020-03-20 13:07:35 -04:00
31c7cecaf9
add a few checks for safety
2020-03-20 10:54:01 -04:00
d386e223e4
simplify open/close of writeStreams
2020-03-19 17:33:22 -04:00
32cd0f3c4d
increase timeout value from 15s to 45. ought to be enough for anybody
2020-03-19 16:13:24 -04:00
05a4e86cdb
close streams whenever we finish using them. time out if necessary
2020-03-19 16:11:24 -04:00
4522ffa18a
compute indexes in child processes
2020-03-19 10:46:18 -04:00
f345998956
fix undefined reference
2020-03-17 10:10:20 -04:00
30f17040ac
close streams when possible, simplify some code, enforce asynchrony
2020-03-16 16:13:38 -04:00
e1a0daac9c
possible server fixes
2020-03-12 11:23:43 -04:00
d4038344ba
stop iterating over tasks if you can't read the root directory
2020-03-11 16:34:00 -04:00
32d769447a
begin standardizing our method of streaming lines from files
2020-03-04 11:38:14 -05:00
35eca2c5d2
pull file streaming out into its own file, leave a few notes
2020-03-04 11:38:14 -05:00
ff73e96cb8
reimplement the trim history fix from staging
2020-02-17 12:02:49 -05:00
725d10fc60
nest storage directory inside './lib'
2020-02-14 16:29:30 -05:00
ansuz