DSM 6.x und darunter BTRFS - zu wenig Platz für Metadaten / KB

[Uwisch]

Hi,

Ich habe bzw hatte ein Problem mit meiner 218+ (mit BTRFS Volume)
- VMM mit einer kleinen VM: hängt sich immer wieder auf
- Hänger beim Datenzugriff

1TB von 8TB sind frei laut GUI.
Nachdem ich Daten und SnapShots gelöscht hatte (nun 2TB frei), ist alles ok - soweit - seit 3 Tagen.

Bei Synology bin ich auf diesen KB gestoßen:
https://kb.synology.com/en-us/DSM/tutorial/Volume_running_low_on_metadata_capacity

Weiß jemand wie ein Volllaufen der Metadaten bei DSM 6.2 festgestellt werden kann?
Welche Log-Einträge in /va/log/messages sind zu erwarten beim Volllaufen?

Danke, Uwe

 

[Adama]

Kannst Du dich mit SSH auf deiner Maschine anmelden?

Wenn ja, kannst Du die Werte mit

btrfs filesystem df /volume1/

abfragen. Du musst natürlich, wenn mehr als ein Volume vorhanden, das entsprechende angeben.

Bei mir z.B. sieht das dann so aus:

Data, single: total=11.42TiB, used=10.39TiB
System, DUP: total=8.00MiB, used=224.00KiB
Metadata, DUP: total=223.00GiB, used=13.68GiB
GlobalReserve, single: total=2.00GiB, used=0.00B

Zu Logeinträgen kann ich nichts sagen, da ich an derartige Grenzen bisher nicht gestossen bin...

 

[Uwisch]

Danke - den Output habe ich protokolliert ;-)

Problem aktiv:
btrfs fi df /volume1/
Data, single: total=6.31TiB, used=4.96TiB
System, DUP: total=8.00MiB, used=224.00KiB
Metadata, DUP: total=488.70GiB, used=481.25GiB
GlobalReserve, single: total=2.00GiB, used=0.00B

Status jetzt - scheinbar keine Probleme:
ash-4.3# btrfs fi df /volume1/
Data, single: total=6.31TiB, used=4.33TiB
System, DUP: total=8.00MiB, used=240.00KiB
Metadata, DUP: total=493.70GiB, used=202.25GiB
GlobalReserve, single: total=2.00GiB, used=41.19MiB

 

[Adama]

Ah ok, dann kennst Du das schon...

Ich hatte dabei dran gedacht, das als Basis für ein Skript zu nehmen, welches dann den Zustand meldet...

 

[Uwisch]

gute Idee - oder auf DSM 7 updaten - dann wird eine Warnung generiert. Aber da zögere ich noch ...

 

[Adama]

Für die Metadaten?

Muss ich direkt mal schauen...

 

[Uwisch]

Nun lief die DS fast 4 Tage ohne auffällige Logeinträge (messages).
Heute wieder ein "blocked" Process:

[434159.632910] INFO: task synosharesnapsh:15272 blocked for more than 120 seconds.
[434159.641190] Tainted: P O 4.4.59+ #25556
[434159.648538] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[434159.658077] synosharesnapsh D ffff880066a0fdf8 0 15272 15208 0x00000000
[434159.666111] ffff880066a0fdf8 0000000061716678 ffff88006b3ec400 ffff880066a10000
[434159.674513] ffff8802617166a8 ffff880261716378 ffff880066a0fe7c ffff880252c1a870
[434159.682892] ffff880066a0fe10 ffffffff81574ec7 ffff880066a0fe7c ffff880066a0fe58
[434159.691326] Call Trace:
[434159.694159] [<ffffffff81574ec7>] schedule+0x37/0x80
[434159.699801] [<ffffffff81164ce5>] wb_wait_for_completion+0x35/0x70
[434159.706803] [<ffffffff81080150>] ? wait_woken+0x80/0x80
[434159.712823] [<ffffffff811655c4>] sync_inodes_sb+0x94/0x1c0
[434159.719144] [<ffffffff8116ad10>] ? SyS_tee+0x3d0/0x3d0
[434159.725085] [<ffffffff8116ad24>] sync_inodes_one_sb+0x14/0x20
[434159.731717] [<ffffffff8113a0ba>] iterate_supers+0xaa/0x100
[434159.738036] [<ffffffff8116b032>] sys_sync+0x32/0x90
[434159.743702] [<ffffffff81578c0a>] entry_SYSCALL_64_fastpath+0x1e/0x92

Könnt ihr dies einschätzen? Während der "Problemzeit" Ende September habe ich sehr viele "blocked" gesehen.

dmesg| grep -i blocked
[61686.692271] INFO: task synoimgbkptool:8677 blocked for more than 120 seconds.
[61806.822012] INFO: task btrfs-transacti:9049 blocked for more than 120 seconds.
[61806.977277] INFO: task synoimgbkptool:8677 blocked for more than 120 seconds.
[63007.048756] INFO: task img_backupd:14595 blocked for more than 120 seconds.
[63127.193314] INFO: task img_backupd:14595 blocked for more than 120 seconds.
[63247.331893] INFO: task img_backupd:14595 blocked for more than 120 seconds.
[63367.472362] INFO: task s2s_daemon:9492 blocked for more than 120 seconds.
[63367.620560] INFO: task img_backupd:14595 blocked for more than 120 seconds.
[63367.766138] INFO: task synosharesnapsh:16257 blocked for more than 120 seconds.
[63367.904959] INFO: task img_backupd:16595 blocked for more than 120 seconds.
[63368.022603] INFO: task synoimgbkptool:16645 blocked for more than 120 seconds.
[63368.164352] INFO: task synoimgbkptool:16649 blocked for more than 120 seconds.
[63368.305618] INFO: task SYNO.CCC.Repo_1:16735 blocked for more than 120 seconds.
[63368.423646] INFO: task img_backupd:16751 blocked for more than 120 seconds.
[63488.558873] INFO: task s2s_daemon:9492 blocked for more than 120 seconds.
[63488.706755] INFO: task img_backupd:14595 blocked for more than 120 seconds.
[63488.852093] INFO: task synosharesnapsh:16257 blocked for more than 120 seconds.
[63488.992702] INFO: task img_backupd:16595 blocked for more than 120 seconds.
[63489.114157] INFO: task synoimgbkptool:16645 blocked for more than 120 seconds.
[63489.262664] INFO: task synoimgbkptool:16649 blocked for more than 120 seconds.
[63489.404646] INFO: task SYNO.CCC.Repo_1:16735 blocked for more than 120 seconds.
[63489.523633] INFO: task img_backupd:16751 blocked for more than 120 seconds.
[63609.654386] INFO: task s2s_daemon:9492 blocked for more than 120 seconds.
[63609.801207] INFO: task img_backupd:14595 blocked for more than 120 seconds.
[63609.947455] INFO: task synosharesnapsh:16257 blocked for more than 120 seconds.
[63610.086492] INFO: task img_backupd:16595 blocked for more than 120 seconds.
[63610.203954] INFO: task synoimgbkptool:16645 blocked for more than 120 seconds.
[63610.343529] INFO: task synoimgbkptool:16649 blocked for more than 120 seconds.
[63610.482187] INFO: task SYNO.CCC.Repo_1:16735 blocked for more than 120 seconds.
[63610.599943] INFO: task img_backupd:16751 blocked for more than 120 seconds.
[63850.724467] INFO: task s2s_daemon:9492 blocked for more than 120 seconds.
[63850.873635] INFO: task img_backupd:14595 blocked for more than 120 seconds.
[63851.011685] INFO: task synosharesnapsh:16257 blocked for more than 120 seconds.
[63851.156994] INFO: task img_backupd:16751 blocked for more than 120 seconds.
[63851.302901] INFO: task synoimgbkptool:17664 blocked for more than 120 seconds.
[63851.449782] INFO: task synoimgbkptool:17739 blocked for more than 120 seconds.
[63851.582849] INFO: task synoimgbkptool:17742 blocked for more than 120 seconds.
[63971.715986] INFO: task s2s_daemon:9492 blocked for more than 120 seconds.
[63971.861312] INFO: task img_backupd:14595 blocked for more than 120 seconds.
[63972.000536] INFO: task synosharesnapsh:16257 blocked for more than 120 seconds.
[63972.148113] INFO: task img_backupd:16751 blocked for more than 120 seconds.
[63972.296563] INFO: task synoimgbkptool:17664 blocked for more than 120 seconds.
[63972.434943] INFO: task synoimgbkptool:17739 blocked for more than 120 seconds.
[63972.566576] INFO: task synoimgbkptool:17742 blocked for more than 120 seconds.
[64092.698497] INFO: task s2s_daemon:9492 blocked for more than 120 seconds.
[64092.846350] INFO: task img_backupd:14595 blocked for more than 120 seconds.
[64092.988650] INFO: task synosharesnapsh:16257 blocked for more than 120 seconds.
[64093.134021] INFO: task img_backupd:16751 blocked for more than 120 seconds.
[64093.279071] INFO: task synoimgbkptool:17664 blocked for more than 120 seconds.
[64093.419623] INFO: task synoimgbkptool:17739 blocked for more than 120 seconds.
[64093.552755] INFO: task synoimgbkptool:17742 blocked for more than 120 seconds.
[64213.688010] INFO: task s2s_daemon:9492 blocked for more than 120 seconds.
[64213.835253] INFO: task img_backupd:14595 blocked for more than 120 seconds.
[64213.972989] INFO: task synosharesnapsh:16257 blocked for more than 120 seconds.
[64214.119303] INFO: task img_backupd:16751 blocked for more than 120 seconds.
[64214.264716] INFO: task synoimgbkptool:17664 blocked for more than 120 seconds.
[64214.404573] INFO: task synoimgbkptool:17739 blocked for more than 120 seconds.
[64214.536523] INFO: task synoimgbkptool:17742 blocked for more than 120 seconds.
[64334.670528] INFO: task s2s_daemon:9492 blocked for more than 120 seconds.
[64574.803656] INFO: task btrfs-transacti:9049 blocked for more than 120 seconds.
[64574.940628] INFO: task s2s_daemon:9492 blocked for more than 120 seconds.
[64575.071640] INFO: task abk-agentless-s:18713 blocked for more than 120 seconds.
[64575.204582] INFO: task img_backupd:14595 blocked for more than 120 seconds.
[64575.356825] INFO: task img_backupd:16751 blocked for more than 120 seconds.
[64575.524987] INFO: task synoimgbkptool:17664 blocked for more than 120 seconds.
[64575.657633] INFO: task synoimgbkptool:17739 blocked for more than 120 seconds.
[64575.802576] INFO: task synoimgbkptool:17740 blocked for more than 120 seconds.
[64575.957475] INFO: task synoimgbkptool:17741 blocked for more than 120 seconds.
[64576.095661] INFO: task synoimgbkptool:17742 blocked for more than 120 seconds.
[64576.240083] INFO: task synoabk-privile:18618 blocked for more than 120 seconds.
[64576.394777] INFO: task btrfs:19687 blocked for more than 120 seconds.
[70576.250761] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[70696.359341] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[70816.482833] INFO: task btrfs-transacti:9049 blocked for more than 120 seconds.
[70816.619538] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[70936.727540] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[71056.858982] INFO: task btrfs-cleaner:9048 blocked for more than 120 seconds.
[71057.045661] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[71177.153530] INFO: task btrfs-cleaner:9048 blocked for more than 120 seconds.
[71177.343982] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[71297.451170] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[71417.559740] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[71537.673333] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[71657.780827] INFO: task btrfs-transacti:9049 blocked for more than 120 seconds.
[71657.919559] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[71778.032393] INFO: task btrfs-transacti:9049 blocked for more than 120 seconds.
[71778.168646] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[71898.278030] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[72018.388623] INFO: task img_backupd:8663 blocked for more than 120 seconds.
[76098.288228] INFO: task synoimgbkp_tagm:3417 blocked for more than 120 seconds.
[76218.426796] INFO: task synoimgbkp_tagm:3417 blocked for more than 120 seconds.
[76338.566374] INFO: task synoimgbkp_tagm:3417 blocked for more than 120 seconds.

 

[Adama]

Hab' grad mal meine /var/log/messages durchsucht, hab' nicht einen Eintrag gefunden.

Heißt das denn, dass deine Metadaten wieder stark angewachsen sind?

By the way, wenn ich auf meinen beiden BTRFS-Synos schaue, dann liegt die 918+ bei 6% Belegung und die 420+ bei 9%...

 

[Uwisch]

Metadaten sind ok.
ash-4.3# btrfs fi df /volume1/
Data, single: total=6.31TiB, used=4.40TiB
System, DUP: total=8.00MiB, used=240.00KiB
Metadata, DUP: total=493.70GiB, used=93.71GiB
GlobalReserve, single: total=2.00GiB, used=0.00B

Ich weiß nicht was da los ist .... GUI alles ok, nichts im Protokoll-Center zu sehen.
Aber wieder aktuell viele "blocked"

ash-4.3# cat /var/log/messages| grep -i blocked


2021-10-07T12:03:50+02:00 DS3-Buero kernel: [434159.632910] INFO: task synosharesnapsh:15272 blocked for more than 120 seconds.


2021-10-08T06:03:50+02:00 DS3-Buero kernel: [498956.293395] INFO: task synosharesnapsh:14204 blocked for more than 120 seconds.


2021-10-08T06:03:50+02:00 DS3-Buero kernel: [498956.414383] INFO: task synosharesnapsh:14206 blocked for more than 120 seconds.


2021-10-08T10:35:50+02:00 DS3-Buero kernel: [515275.661868] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:35:50+02:00 DS3-Buero kernel: [515275.821413] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:37:50+02:00 DS3-Buero kernel: [515396.026420] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:37:50+02:00 DS3-Buero kernel: [515396.184713] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:39:51+02:00 DS3-Buero kernel: [515516.388991] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:39:51+02:00 DS3-Buero kernel: [515516.548432] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:41:51+02:00 DS3-Buero kernel: [515636.753553] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:41:51+02:00 DS3-Buero kernel: [515636.912187] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:43:51+02:00 DS3-Buero kernel: [515757.119113] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:43:52+02:00 DS3-Buero kernel: [515757.278354] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:45:52+02:00 DS3-Buero kernel: [515877.482704] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:45:52+02:00 DS3-Buero kernel: [515877.641995] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:47:52+02:00 DS3-Buero kernel: [515997.850232] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:47:53+02:00 DS3-Buero kernel: [515998.012483] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:49:53+02:00 DS3-Buero kernel: [516118.216813] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:49:53+02:00 DS3-Buero kernel: [516118.374492] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:51:53+02:00 DS3-Buero kernel: [516238.579406] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:51:53+02:00 DS3-Buero kernel: [516238.739456] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:53:53+02:00 DS3-Buero kernel: [516358.966998] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:53:53+02:00 DS3-Buero kernel: [516359.126667] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:55:54+02:00 DS3-Buero kernel: [516479.331583] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:55:54+02:00 DS3-Buero kernel: [516479.490284] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:57:54+02:00 DS3-Buero kernel: [516599.696171] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:57:54+02:00 DS3-Buero kernel: [516599.855464] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T10:59:54+02:00 DS3-Buero kernel: [516720.061483] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T10:59:55+02:00 DS3-Buero kernel: [516720.226318] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T11:01:55+02:00 DS3-Buero kernel: [516840.433343] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T11:01:55+02:00 DS3-Buero kernel: [516840.595067] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T11:03:55+02:00 DS3-Buero kernel: [516960.798928] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T11:03:55+02:00 DS3-Buero kernel: [516960.957944] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T11:05:56+02:00 DS3-Buero kernel: [517081.162512] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T11:05:56+02:00 DS3-Buero kernel: [517081.326088] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T11:07:56+02:00 DS3-Buero kernel: [517201.569095] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T11:07:56+02:00 DS3-Buero kernel: [517201.728322] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T11:09:56+02:00 DS3-Buero kernel: [517321.932702] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T11:09:56+02:00 DS3-Buero kernel: [517322.104253] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T11:11:57+02:00 DS3-Buero kernel: [517442.309260] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T11:11:57+02:00 DS3-Buero kernel: [517442.468471] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T11:13:57+02:00 DS3-Buero kernel: [517562.673845] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T11:13:57+02:00 DS3-Buero kernel: [517562.832668] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T11:15:57+02:00 DS3-Buero kernel: [517683.036427] INFO: task jbd2/sdq1-8:10860 blocked for more than 120 seconds.


2021-10-08T11:15:58+02:00 DS3-Buero kernel: [517683.195945] INFO: task kworker/u8:0:7173 blocked for more than 120 seconds.


2021-10-08T12:03:58+02:00 DS3-Buero kernel: [520563.257933] INFO: task synosharesnapsh:7111 blocked for more than 120 seconds.

 

[Adama]

Ich hab' mal trivial nach "linux blocked for more than 120 seconds" gegoogelt:

Das sind die ersten beiden Ergebnisse, die ich schon ganz interessant finde
https://www.cmo.de/wissensdatenbank/linux-problem-task-blocked-for-more-than-120-seconds/
https://serverfault.com/questions/4...g-and-error-blocked-for-more-than-120-seconds

So ganz schnell aus der Hüfte geschossen, würd ich sagen RAM knapp...

 

[Uwisch]

RAM is kaum ausgelastet - 10GB sind vorhanden, 20% genutzt.
Momentan bin ich ratlos. Die Probleme hatte ich auf vollen Metadaten Platz zurückgeführt.

Metadata, DUP: total=493.70GiB, used=95.54GiB
Hier ist jetzt aber genug frei.
Die Logeinträge bezüglich BTRFS sind auch verschwunden - dennoch gibt es viele der "blocked" Meldungen.
Ich überlege die DS komplett neu aufzusetzen ...

 

[Adama]

Kann ich verstehen, wirkt alles sehr rätselhaft...

 

[Uwisch]

Hallo, ich habe inzwischen (am Donnerstag) die DS komplett gelöscht und DSM 7.0.1 neu installiert. Daten sind zurückgespielt. VM läuft.Backup auf USB-Festplatte läuft.

Heute sehe ich wieder "blocked" Meldungen.
Könnten diese mit der angeschlossenen USB Festplatte zu tun haben? Zumindest ist dort ext4 in Benutzung.

[196069.445618] INFO: task img_worker:6570 blocked for more than 120 seconds.
[196069.453315] Tainted: P O 4.4.180+ #42218
[196069.459682] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[196069.468574] img_worker D ffff880037ebbdc8 0 6570 5138 0x00000000
[196069.476656] ffff880037ebbdc8 0000000000000003 ffff8802f3a08dc0 ffff88022fbc44c0
[196069.485082] ffff8802e0908800 00000000003080ce ffff8802e0908824 ffff8802e0908888
[196069.493495] ffff8802e09088a0 ffff880037ebbdd8 ffffffff81576126 ffff880037ebbe38
[196069.501955] Call Trace:
[196069.504792] [<ffffffff81576126>] schedule+0x26/0x70
[196069.510453] [<ffffffff8127f7bf>] jbd2_log_wait_commit+0x9f/0x100
[196069.517399] [<ffffffff8108e8b0>] ? wake_bit_function+0x60/0x60
[196069.524124] [<ffffffff812826db>] jbd2_complete_transaction+0x5b/0xa0
[196069.531441] [<ffffffff812175c3>] ext4_sync_file+0x303/0x360
[196069.537885] [<ffffffff811e2bb1>] ? locks_free_lock+0x41/0x50
[196069.544425] [<ffffffff811c88c9>] vfs_fsync_range+0x49/0xa0
[196069.550766] [<ffffffff81163eb5>] ? find_vma+0x65/0x70
[196069.556643] [<ffffffff811699bd>] SyS_msync+0x17d/0x210
[196069.562595] [<ffffffff81579c4a>] entry_SYSCALL_64_fastpath+0x1e/0x8e
[201709.245397] INFO: task img_worker:6568 blocked for more than 120 seconds.
[201709.253100] Tainted: P O 4.4.180+ #42218
[201709.259466] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[201709.268373] img_worker D ffff8800403c7dc8 0 6568 5138 0x00000000
[201709.276389] ffff8800403c7dc8 0000000000000003 ffff8802be28d280 ffff88022fbc1b80
[201709.284858] ffff8802e0908800 00000000003081c4 ffff8802e0908824 ffff8802e0908888
[201709.293316] ffff8802e09088a0 ffff8800403c7dd8 ffffffff81576126 ffff8800403c7e38
[201709.301754] Call Trace:
[201709.304592] [<ffffffff81576126>] schedule+0x26/0x70
[201709.310250] [<ffffffff8127f7bf>] jbd2_log_wait_commit+0x9f/0x100
[201709.317175] [<ffffffff8108e8b0>] ? wake_bit_function+0x60/0x60
[201709.323906] [<ffffffff812826db>] jbd2_complete_transaction+0x5b/0xa0
[201709.331243] [<ffffffff812175c3>] ext4_sync_file+0x303/0x360
[201709.337694] [<ffffffff811e2bb1>] ? locks_free_lock+0x41/0x50
[201709.344267] [<ffffffff811c88c9>] vfs_fsync_range+0x49/0xa0
[201709.350608] [<ffffffff81163e67>] ? find_vma+0x17/0x70
[201709.356464] [<ffffffff811699bd>] SyS_msync+0x17d/0x210
[201709.362411] [<ffffffff81579c4a>] entry_SYSCALL_64_fastpath+0x1e/0x8e
[201709.369808] INFO: task img_worker:6570 blocked for more than 120 seconds.
[201709.377511] Tainted: P O 4.4.180+ #42218
[201709.383857] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
[201709.392752] img_worker D ffff880037ebbdc8 0 6570 5138 0x00000000
[201709.400801] ffff880037ebbdc8 0000000000000003 ffff8802f3a08dc0 ffff88022fbc44c0
[201709.409190] ffff8802e0908800 00000000003081c5 ffff8802e0908824 ffff8802e0908888
[201709.417660] ffff8802e09088a0 ffff880037ebbdd8 ffffffff81576126 ffff880037ebbe38
[201709.426119] Call Trace:
[201709.428954] [<ffffffff81576126>] schedule+0x26/0x70
[201709.434616] [<ffffffff8127f7bf>] jbd2_log_wait_commit+0x9f/0x100
[201709.441555] [<ffffffff8108e8b0>] ? wake_bit_function+0x60/0x60
[201709.448282] [<ffffffff812826db>] jbd2_complete_transaction+0x5b/0xa0
[201709.455603] [<ffffffff812175c3>] ext4_sync_file+0x303/0x360
[201709.462043] [<ffffffff811e2bb1>] ? locks_free_lock+0x41/0x50
[201709.468591] [<ffffffff811c88c9>] vfs_fsync_range+0x49/0xa0
[201709.474931] [<ffffffff81163e67>] ? find_vma+0x17/0x70
[201709.480788] [<ffffffff811699bd>] SyS_msync+0x17d/0x210
[201709.486731] [<ffffffff81579c4a>] entry_SYSCALL_64_fastpath+0x1e/0x8e