Issue Tracker

Issue Log

Failed to allocate directory watch: Too many open files

Noticing issues on master1a, seeing this error message when trying to install tools.

Failed to allocate directory watch: Too many open files

Also whilst inspecting the dmesg

[1039859.270706] EXT4-fs error (device sdc) in ext4_reserve_inode_write:5812: Journal has aborted [1039859.270721] EXT4-fs error (device sdc) in ext4_reserve_inode_write:5812: Journal has aborted [1039859.270725] EXT4-fs error (device sdc): ext4_handle_inode_extension:319: inode #1449868: comm minio: mark_inode_dirty error [1039859.270728] EXT4-fs error (device sdc) in ext4_handle_inode_extension:321: Journal has aborted [1039859.270731] EXT4-fs error (device sdc): ext4_journal_check_start:84: comm minio: Detected aborted journal [1039859.327640] sd 2:0:0:1: [sdc] tag#84 Sense Key : 0x3 [current] [1039859.327644] sd 2:0:0:1: [sdc] tag#84 ASC=0x11 ASCQ=0x0 [1039859.327646] sd 2:0:0:1: [sdc] tag#84 CDB: opcode=0x28 28 00 00 4b d2 80 00 00 20 00 [1039864.032370] EXT4-fs (sdc): Remounting filesystem read-only [1039864.032480] EXT4-fs (sdc): Remounting filesystem read-only

This is a result of the filesystem being in a read-only state. This can happen if the system detects an error with the filesystem, such as a corrupted journal or a hardware issue with the disk.

After a sereis of googling and ChatGPT questions, the next thing I will try is

sudo sysctl -w fs.inotify.max_user_watches=262144 sudo sysctl -w fs.inotify.max_user_instances=256 # Double it for good measure

we now wait, see if the error messages in alert manger drop off.

Update 12/04 - This seemed to have fixed the error.

Collapsed by default

This is the definition of the second term.