Navigation

    Logo
    • Register
    • Login
    • Search
    • Recent
    • Tags
    • Unread
    • Categories
    • Unreplied
    • Popular
    • GitHub
    • Docu
    • Hilfe
    1. Home
    2. Deutsch
    3. Error/Bug
    4. [gelöst] objects.jsonl wächst rasant

    NEWS

    • Neuer Blog: Fotos und Eindrücke aus Solingen

    • ioBroker@Smart Living Forum Solingen, 14.06. - Agenda added

    • ioBroker goes Matter ... Matter Adapter in Stable

    [gelöst] objects.jsonl wächst rasant

    This topic has been deleted. Only users with topic management privileges can see it.
    • B
      BiberM last edited by BiberM

      Systemdata Bitte Ausfüllen
      Hardwaresystem: Raspi 4b + Conbee II Stick
      Arbeitsspeicher: 4GB
      Festplattenart: 32GB MicroSD
      Betriebssystem: Raspian Bullseye
      Node-Version: 14.19.3
      Nodejs-Version: 14.19.3
      NPM-Version: 6.14.17
      Installationsart: Skript
      Image genutzt: Nein
      Ort/Name der Imagedatei:

      Guten Abend,

      ich habe seit einigen Tagen das Problem, dass meine objects.jsonl sehr schnell wächst (ca. 1 GB / Tag).

      pi@smarthomepi:~ $ iob list instances
      + system.adapter.admin.0                  : admin                 : smarthomepi                              -  enabled, port: 8081, bind: 0.0.0.0, run as: admin
        system.adapter.backitup.0               : backitup              : smarthomepi                              - disabled
      + system.adapter.deconz.0                 : deconz                : smarthomepi                              -  enabled
        system.adapter.discovery.0              : discovery             : smarthomepi                              - disabled
        system.adapter.fritzdect.0              : fritzdect             : smarthomepi                              - disabled
        system.adapter.history.0                : history               : smarthomepi                              - disabled
        system.adapter.icons-open-icon-library-png.0: icons-open-icon-library-png: smarthomepi                              - disabled
        system.adapter.info.0                   : info                  : smarthomepi                              - disabled
      + system.adapter.javascript.0             : javascript            : smarthomepi                              -  enabled
        system.adapter.material.0               : material              : smarthomepi                              - disabled
        system.adapter.rpi2.0                   : rpi2                  : smarthomepi                              - disabled
      + system.adapter.shelly.1                 : shelly                : smarthomepi                              -  enabled, port: 1882, bind: 0.0.0.0
        system.adapter.vis-bars.0               : vis-bars              : smarthomepi                              - disabled
        system.adapter.vis-colorpicker.0        : vis-colorpicker       : smarthomepi                              - disabled
        system.adapter.vis-fancyswitch.0        : vis-fancyswitch       : smarthomepi                              - disabled
        system.adapter.vis-hqwidgets.0          : vis-hqwidgets         : smarthomepi                              - disabled
        system.adapter.vis-justgage.0           : vis-justgage          : smarthomepi                              - disabled
        system.adapter.vis-plumb.0              : vis-plumb             : smarthomepi                              - disabled
        system.adapter.vis.0                    : vis                   : smarthomepi                              -  enabled
      + system.adapter.web.0                    : web                   : smarthomepi                              -  enabled, port: 8082, bind: 0.0.0.0, run as: admin
      
      + instance is alive
      
      pi@smarthomepi:~ $ iob list adapters
      system.adapter.admin                   : admin          - v5.3.8
      system.adapter.backitup                : backitup       - v2.4.9
      system.adapter.deconz                  : deconz         - v1.3.21
      system.adapter.discovery               : discovery      - v3.0.3
      system.adapter.fritzdect               : fritzdect      - v2.2.3
      system.adapter.history                 : history        - v1.11.1
      system.adapter.icons-open-icon-library-png: icons-open-icon-library-png - v0.1.2
      system.adapter.info                    : info           - v1.9.19
      system.adapter.javascript              : javascript     - v5.7.0
      system.adapter.material                : material       - v1.1.2
      system.adapter.rpi2                    : rpi2           - v1.3.2
      system.adapter.shelly                  : shelly         - v5.3.2
      system.adapter.simple-api              : simple-api     - v2.6.1
      system.adapter.socketio                : socketio       - v4.2.0
      system.adapter.sql                     : sql            - v1.15.7
      system.adapter.vis                     : vis            - v1.4.15
      system.adapter.vis-bars                : vis-bars       - v0.1.4
      system.adapter.vis-colorpicker         : vis-colorpicker - v1.2.0
      system.adapter.vis-fancyswitch         : vis-fancyswitch - v1.1.0
      system.adapter.vis-hqwidgets           : vis-hqwidgets  - v1.2.0
      system.adapter.vis-justgage            : vis-justgage   - v1.0.2
      system.adapter.vis-plumb               : vis-plumb      - v1.0.2
      system.adapter.web                     : web            - v4.3.0
      

      2 aufeinanderfolgende ls -al innerhalb weniger Minuten sahen wie folgt aus

      pi@smarthomepi:~ $ ls -al /opt/iobroker/iobroker-data/
      insgesamt 44168
      drwxrwxr-x+  8 iobroker iobroker     4096 13. Jun 22:02 .
      drwxrwxr-x+  6 iobroker iobroker     4096 13. Jun 22:23 ..
      drwxrwxr-x+  2 iobroker iobroker     4096 13. Jun 20:22 backitup
      drwxrwxr-x+  2 iobroker iobroker    12288 13. Jun 17:56 backup-objects
      drwxrwxr-x+ 28 iobroker iobroker     4096 13. Jun 22:23 files
      drwxrwxr-x+ 14 iobroker iobroker     4096 13. Jun 21:47 history
      -rw-rwxr--+  1 iobroker iobroker     4425 13. Jun 22:38 iobroker.json
      -rw-rwxr--+  1 iobroker iobroker        3 13. Jun 21:49 notifications.json
      -rw-rwxr--+  1 iobroker iobroker  7657352 17. Mär 21:40 objects.json.bak.migrated
      -rw-rw-r--+  1 iobroker iobroker 25530073 13. Jun 22:40 objects.jsonl
      drwxrwxr-x+  2 iobroker iobroker     4096 13. Jun 22:41 objects.jsonl.lock
      -rw-rwxr--+  1 iobroker iobroker  7657352 17. Mär 21:40 objects.json.migrated
      -rw-rwxr--+  1 iobroker iobroker   837073 17. Mär 21:39 states.json.bak.migrated
      -rw-rw-r--+  1 iobroker iobroker  2641643 13. Jun 22:41 states.jsonl
      drwxrwxr-x+  2 iobroker iobroker     4096 13. Jun 22:41 states.jsonl.lock
      -rw-rwxr--+  1 iobroker iobroker   837102 17. Mär 21:40 states.json.migrated
      pi@smarthomepi:~ $ ls -al /opt/iobroker/iobroker-data/
      insgesamt 44208
      drwxrwxr-x+  8 iobroker iobroker     4096 13. Jun 22:02 .
      drwxrwxr-x+  6 iobroker iobroker     4096 13. Jun 22:23 ..
      drwxrwxr-x+  2 iobroker iobroker     4096 13. Jun 20:22 backitup
      drwxrwxr-x+  2 iobroker iobroker    12288 13. Jun 17:56 backup-objects
      drwxrwxr-x+ 28 iobroker iobroker     4096 13. Jun 22:23 files
      drwxrwxr-x+ 14 iobroker iobroker     4096 13. Jun 21:47 history
      -rw-rwxr--+  1 iobroker iobroker     4425 13. Jun 22:38 iobroker.json
      -rw-rwxr--+  1 iobroker iobroker        3 13. Jun 21:49 notifications.json
      -rw-rwxr--+  1 iobroker iobroker  7657352 17. Mär 21:40 objects.json.bak.migrated
      -rw-rw-r--+  1 iobroker iobroker 25552652 13. Jun 22:41 objects.jsonl
      drwxrwxr-x+  2 iobroker iobroker     4096 13. Jun 22:43 objects.jsonl.lock
      -rw-rwxr--+  1 iobroker iobroker  7657352 17. Mär 21:40 objects.json.migrated
      -rw-rwxr--+  1 iobroker iobroker   837073 17. Mär 21:39 states.json.bak.migrated
      -rw-rw-r--+  1 iobroker iobroker  2655495 13. Jun 22:42 states.jsonl
      drwxrwxr-x+  2 iobroker iobroker     4096 13. Jun 22:43 states.jsonl.lock
      -rw-rwxr--+  1 iobroker iobroker   837102 17. Mär 21:40 states.json.migrated
      

      Ich hatte zudem letztens Fehler, dass iobroker update nicht wollte, weil auf die Datenbank nicht mehr zugegriffen werden konnte. Ich habe Backups der objects und states eingespielt und es ging wieder alles. Bis diese wieder anfingen start zu wachsen (objects deutlich stärker).

      Ich habe den Thread link gefunden. Die hier angesprochenen Einstellungen sind bei mir in der iobroker.json nicht vorhanden, trotzdem scheint keine Komprimierung stattzufinden.

      Hat noch jemand eine Idee, woran das liegen könnte? Welche Informationen werden noch benötigt?

      Vielen Dank im Voraus!
      Michael

      Thomas Braun AlCalzone 2 Replies Last reply Reply Quote 0
      • Thomas Braun
        Thomas Braun Most Active @BiberM last edited by

        @biberm

        Bin zu faul umzurechnen...
        Bitte mal -h setzen:

        ls -lh /opt/iobroker/iobroker-data/
        
        B 1 Reply Last reply Reply Quote 0
        • AlCalzone
          AlCalzone Developer @BiberM last edited by

          @biberm sagte in objects.jsonl wächst rasant:

          Welche Informationen werden noch benötigt?

          Welche JS-Controller-Version?
          Wie viele Objekte hast du laut ioBroker Admin?
          Wie genau sieht deine iobroker.json aus?
          Wie viele Zeilen sind in der objects.jsonl? wc -l < /opt/iobroker/iobroker-data/objects.jsonl

          B 1 Reply Last reply Reply Quote 0
          • B
            BiberM @Thomas Braun last edited by

            @thomas-braun Oh Gott, es war schon zu spät gestern. Der Hinweis ist gar nicht so verkehrt. Sind keine GB sondern MB

            pi@smarthomepi:~ $ ls -lh /opt/iobroker/iobroker-data/
            insgesamt 51M
            drwxrwxr-x+  2 iobroker iobroker 4,0K 13. Jun 20:22 backitup
            drwxrwxr-x+  2 iobroker iobroker  12K 14. Jun 21:52 backup-objects
            drwxrwxr-x+ 28 iobroker iobroker 4,0K 13. Jun 22:23 files
            drwxrwxr-x+ 14 iobroker iobroker 4,0K 13. Jun 21:47 history
            -rw-rwxr--+  1 iobroker iobroker 4,4K 13. Jun 22:38 iobroker.json
            -rw-rwxr--+  1 iobroker iobroker    3 13. Jun 21:49 notifications.json
            -rw-rwxr--+  1 iobroker iobroker 7,4M 17. Mär 21:40 objects.json.bak.migrated
            -rw-rw-r--+  1 iobroker iobroker  30M 14. Jun 22:32 objects.jsonl
            drwxrwxr-x+  2 iobroker iobroker 4,0K 14. Jun 22:49 objects.jsonl.lock
            -rw-rwxr--+  1 iobroker iobroker 7,4M 17. Mär 21:40 objects.json.migrated
            -rw-rwxr--+  1 iobroker iobroker 818K 17. Mär 21:39 states.json.bak.migrated
            -rw-rw-r--+  1 iobroker iobroker 5,2M 14. Jun 22:49 states.jsonl
            drwxrwxr-x+  2 iobroker iobroker 4,0K 14. Jun 22:49 states.jsonl.lock
            -rw-rwxr--+  1 iobroker iobroker 818K 17. Mär 21:40 states.json.migrated
            

            Damit wahrscheinlich größentechnisch eher unspannend, oder?

            1 Reply Last reply Reply Quote 0
            • B
              BiberM @AlCalzone last edited by

              @alcalzone
              JS-Controller: 4.0.23
              Objekte: 2398, Zustände: 2036
              Zeilen in objects.jsonl: 6590

              iobroker.json:

              pi@smarthomepi:~ $ cat /opt/iobroker/iobroker-data/iobroker.json
              {
                "system": {
                  "memoryLimitMB": 0,
                  "hostname": "",
                  "statisticsInterval": 15000,
                  "statisticsIntervalComment": "Interval how often the counters for input/output in adapters and controller will be updated",
                  "checkDiskInterval": 300000,
                  "checkDiskIntervalComment": "Interval how often the disk size will be checked",
                  "noChmod": false,
                  "instanceStartInterval": 2000,
                  "noChmodComment": "Flag to test new feature with no chmod call. Must be deleted later and noChmod must be mainline (2018.06.04)",
                  "compact": false,
                  "compactComment": "Controller will try to start the instances as a part of the same process. No spawn will be done. Only by adapters that support it and have flag compact flag in io-package.json",
                  "allowShellCommands": false,
                  "allowShellCommandsComment": "Allow execution of \"shell\" sendToHost commands"
                },
                "multihostService": {
                  "enabled": false,
                  "secure": true
                },
                "network": {
                  "IPv4": true,
                  "IPv6": true,
                  "bindAddress": null
                },
                "objects": {
                  "type": "jsonl",
                  "typeComment": "Possible values: 'file' - [port 9001], redis - [port 6379], couch - [port 5984].",
                  "host": "127.0.0.1",
                  "port": 9001,
                  "user": "",
                  "pass": "",
                  "noFileCache": false,
                  "connectTimeout": 2000,
                  "options": {
                    "auth_pass": null,
                    "retry_max_delay": 5000
                  },
                  "backup": {
                    "disabled": false,
                    "files": 24,
                    "filesComment": "Minimal number of backup files, after the deletion will be executed according to backupTime settings",
                    "hours": 48,
                    "hoursComment": "All backups older than 48 hours will be deleted. But only if the number of files is greater than of backupNumber",
                    "period": 120,
                    "periodComment": "by default backup every 2 hours. Time is in minutes. To disable backup set the value to 0",
                    "path": "",
                    "pathComment": "Absolute path to backup directory or empty to backup in data directory"
                  },
                  "dataDir": "../../iobroker-data/"
                },
                "states": {
                  "type": "jsonl",
                  "typeComment": "Possible values: 'file' - [port 9000], 'redis' - [port 6379].",
                  "host": "127.0.0.1",
                  "port": 9000,
                  "maxQueue": 1000,
                  "user": "",
                  "pass": "",
                  "connectTimeout": 2000,
                  "options": {
                    "auth_pass": null,
                    "retry_max_delay": 5000
                  },
                  "backup": {
                    "disabled": false,
                    "files": 24,
                    "filesComment": "Minimal number of backup files, after the deletion will be executed according to backupTime settings",
                    "hours": 48,
                    "hoursComment": "All backups older than 48 hours will be deleted. But only if the number of files is greater than of backupNumber",
                    "period": 120,
                    "periodComment": "by default backup every 2 hours. Time is in minutes. To disable backup set the value to 0",
                    "path": "",
                    "pathComment": "Absolute path to backup directory or empty to backup in data directory"
                  },
                  "dataDir": "../../iobroker-data/"
                },
                "log": {
                  "level": "info",
                  "maxDays": 7,
                  "noStdout": true,
                  "transport": {
                    "file1": {
                      "type": "file",
                      "enabled": true,
                      "filename": "log/iobroker",
                      "fileext": ".log",
                      "maxSize": null,
                      "maxFiles": null
                    },
                    "syslog1": {
                      "type": "syslog",
                      "enabled": false,
                      "host": "localhost",
                      "hostComment": "The host running syslogd, defaults to localhost.",
                      "portComment": "The port on the host that syslog is running on, defaults to syslogd's default port(514/UDP).",
                      "protocol": "udp4",
                      "protocolComment": "The network protocol to log over (e.g. tcp4, udp4, unix, unix-connect, etc).",
                      "pathComment": "The path to the syslog dgram socket (i.e. /dev/log or /var/run/syslog for OS X).",
                      "facilityComment": "Syslog facility to use (Default: local0).",
                      "localhost": "iobroker",
                      "localhostComment": "Host to indicate that log messages are coming from (Default: localhost).",
                      "sysLogTypeComment": "The type of the syslog protocol to use (Default: BSD).",
                      "app_nameComment": "The name of the application (Default: process.title).",
                      "eolComment": "The end of line character to be added to the end of the message (Default: Message without modifications)."
                    }
                  }
                },
                "dataDirComment": "Always relative to iobroker.js-controller/",
                "dataDir": "../../iobroker-data/"
              

              Aktuell läuft iob update / upgrade wieder problemlos. Ich habe gestern noch ein paar unbenutzte Adapter entfernt.
              Zudem habe ich eben auch schon geantwortet, dass ich mich um eine Größenordnung verrechnet habe. Damit dürfte die Größe als Fehlerquelle für nicht sperrbare Datenbank / nicht zugreifbare Datenbank ausscheiden. Ich werde das mal weiter beobachten. Danke euch beiden für den Schubs in die richtige Richtung.

              AlCalzone 1 Reply Last reply Reply Quote 0
              • AlCalzone
                AlCalzone Developer @BiberM last edited by

                @biberm oh ich dachte auch das sind GB. So ist das eher unkritisch. Wenn sie nicht gelockt werden kann heißt das auch eher dass noch ein Prozess läuft der das Lock hält.

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post

                Support us

                ioBroker
                Community Adapters
                Donate

                373
                Online

                31.8k
                Users

                80.0k
                Topics

                1.3m
                Posts

                [problem] jsonl
                3
                6
                187
                Loading More Posts
                • Oldest to Newest
                • Newest to Oldest
                • Most Votes
                Reply
                • Reply as topic
                Log in to reply
                Community
                Impressum | Datenschutz-Bestimmungen | Nutzungsbedingungen
                The ioBroker Community 2014-2023
                logo