Skip to content
  • mad's avatar
    Validate ota crawler (#1187) · 0d43307f
    mad authored
    * starts implementation of engine
    
    * working implementation of request to node server
    
    * includes net::http implementation if preferred
    
    * implementation of node server in app - not working
    
    * disable Rack Attack
    
    * seed DocumentTypes
    
    * schema changes in dev DO NOT MERGE
    
    * Document how to see document_types
    
    * Add ota server
    
    * image: pondersource/tosdr-ota:1.0
    
    * ota image v1.1
    
    * Use tosdr-ota:1.2
    
    * Ref https://pptr.dev/guides/docker#usage
    
    , fix #1175
    
    * Use pondersource/tosdr-ota:1.3
    
    * manages ota env variables
    
    * makes flash errors more robust
    
    * implements ota crawler functionality, error handling for relevant methods
    
    * updates views to account for new ota crawler
    
    * removes outdated references to old crawlers
    
    * removes ota engine dependency -- no longer needed!
    
    * removes useless code
    
    * removes yarn dependencies for ota engine
    
    * clarifies standard error
    
    * resolves conflict
    
    * updates postgres to 14 to bring in line with produciton
    
    * upgrades postgres in dev
    
    * configures database for proper simulation of production env
    
    * implements way to restore elasticsearch if documents missing for annotations
    
    * adds last crawl date to documents
    
    * adds es volumes
    
    * refactor
    
    * handles missing points after new crawl
    
    * validates and converts selector
    
    * implements way to restore annotations in es
    
    * comments out useless code
    
    ---------
    
    Co-authored-by: default avatarMichiel de Jong <michiel@pondersource.com>
    0d43307f