Преглед изворни кода

add batch process explanation

david пре 7 година
родитељ
комит
6c3d78d193
1 измењених фајлова са 22 додато и 4 уклоњено
  1. 22 4
      README.md

+ 22 - 4
README.md

@@ -1,7 +1,3 @@
-# CLI
-
-To be re-implemented
-
 # Requirements
 
 * S3 (minio can be used for a local instance).
@@ -9,6 +5,24 @@ To be re-implemented
 * Postgresql
 * Python3.6
 
+# Batch process
+
+To remove the initial roundtrip when looking for books, I keep a cache in postgres.  
+To populate this cache, I triggered each of the bot's `LIST` command, at `#ebooks`
+
+I executed:
+
+```
+@pondering42
+@dv8
+@shytot
+@dragnbreaker
+@Xon-new
+```
+
+Then fetched all of the files, unarchived and concatenated them together.
+On the resulting file I ran `parse.py` which inserts it into postgres.
+
 # Services
 
 ## Installation
@@ -55,3 +69,7 @@ if ($http_user_agent ~* "armv7l") {
     rewrite ^/(.*)$ /books/kindle;
 }
 ```
+
+# CLI
+To be re-implemented
+