Commit Graph

40 Commits (f09ef072ac49743570be3d8d89d6ec5755650ae7)

Author SHA1 Message Date
yflory ea4c529e39 Remove offset code 2021-02-09 15:29:34 +01:00
yflory 0fc8bfde4a Use new server format for history keeper time 2021-01-20 15:06:43 +01:00
ansuz de90949e3e suppress ENOENT on metadata when restoring archived files 2020-12-11 20:12:36 +05:30
David Benqué 5e7eb4cbf8 Merge branch 'soon' into staging 2020-12-04 13:39:47 +00:00
ansuz d4e6d86961 enforce asynchrony in new database offset methods 2020-11-27 08:21:51 +05:30
ansuz 466072d03b read files starting from the oldest known point of relevance when computing indices 2020-11-26 12:15:02 +05:30
ansuz cd77b5ffc8 Merge branch 'soon' into staging 2020-11-19 13:18:08 +05:30
yflory bd4b44476d Archive and restore UI in the admin panel 2020-11-18 16:49:40 +01:00
ansuz e185008846 complete uploads in child processes
also fix a nasty race condition for unowned file uploads
2020-11-18 18:26:21 +05:30
ansuz d72e053560 make a note to improve stream timeout error handling 2020-10-22 11:24:58 +05:30
ansuz 67430de7ff Make efforts to avoid closing streams mid-read
1. Close streams when you're done with them
2. Close streams if they seem to have been abandoned
2020-10-22 11:17:03 +05:30
ansuz e8428a2a73 prevent a case of multiple callbacks 2020-10-21 18:13:10 +05:30
ansuz 6e57366b7f address a file descriptor leak 2020-10-13 13:14:39 +05:30
ansuz 46ebd7b40b remove unsupported storage configuration parameters 2020-10-12 17:40:21 +05:30
ansuz 5358fab615 more WIP inactive account eviction 2020-09-29 18:04:51 +05:30
ansuz d794e0b48f include inactive accounts in the archival script 2020-09-24 19:02:22 +05:30
ansuz b04133e6c3 remove some dead code 2020-07-23 09:28:00 -04:00
ansuz fd169ff39c close streams after 2 minutes instead of 5
Hopefully this helps to even out our memory and CPU usage.
There's also some improved labels so that IF streams time out
we will know the exact origin of the offending call.
2020-04-17 15:05:48 -04:00
ansuz 7fac997e93 increase some file storage timeouts related to streams 2020-04-08 09:38:48 -04:00
ansuz d802173325 remove an invalid line 2020-04-06 10:18:11 -04:00
ansuz adfd1eaaa2 avoid a silly typeError in the event of an error 2020-04-02 22:57:57 -04:00
ansuz bc13a21796 give fileStreams a little bit more time before closing them 2020-03-25 11:50:45 -04:00
ansuz c2fcba31e3 remove an XXX 2020-03-24 15:02:46 -04:00
ansuz e128683ffb call back with errors instead of throwing within the Storage module 2020-03-24 13:31:03 -04:00
yflory dc0b26da0b Merge branch 'soon' into staging 2020-03-23 12:33:17 +01:00
ansuz ed63d98ab0 Merge branch 'scaling-index' into soon 2020-03-20 18:57:51 -04:00
ansuz 590e249500 resolve merge conflict 2020-03-20 13:07:35 -04:00
ansuz 31c7cecaf9 add a few checks for safety 2020-03-20 10:54:01 -04:00
ansuz d386e223e4 simplify open/close of writeStreams 2020-03-19 17:33:22 -04:00
ansuz 32cd0f3c4d increase timeout value from 15s to 45. ought to be enough for anybody 2020-03-19 16:13:24 -04:00
ansuz 05a4e86cdb close streams whenever we finish using them. time out if necessary 2020-03-19 16:11:24 -04:00
ansuz 4522ffa18a compute indexes in child processes 2020-03-19 10:46:18 -04:00
ansuz f345998956 fix undefined reference 2020-03-17 10:10:20 -04:00
ansuz 30f17040ac close streams when possible, simplify some code, enforce asynchrony 2020-03-16 16:13:38 -04:00
ansuz e1a0daac9c possible server fixes 2020-03-12 11:23:43 -04:00
ansuz d4038344ba stop iterating over tasks if you can't read the root directory 2020-03-11 16:34:00 -04:00
ansuz 32d769447a begin standardizing our method of streaming lines from files 2020-03-04 11:38:14 -05:00
ansuz 35eca2c5d2 pull file streaming out into its own file, leave a few notes 2020-03-04 11:38:14 -05:00
ansuz ff73e96cb8 reimplement the trim history fix from staging 2020-02-17 12:02:49 -05:00
ansuz 725d10fc60 nest storage directory inside './lib' 2020-02-14 16:29:30 -05:00