我们开始出错了
open /var/lib/docker/overlay/-init/merged/dev/console: cannot allocate memory
我们得到了
page allocation failure: order:4
运行页面缓存转储会暂时解决问题
echo 1 > /proc/sys/vm/drop_caches
所以我注意到在运行docker任务时,Dirty内存块峰值(应该如此)和Mapped跳过它.但是,DirectMap4k相对接近那个跳跃.
例如:
闲置机器
cat /proc/meminfo | grep -P "(Dirty|Mapped|DirectMap4k)"
Dirty: 104 kB
Mapped: 45696 kB
DirectMap4k: 100352 kB
主动机器
cat /proc/meminfo | grep -P "(Dirty|Mapped|DirectMap4k)"
Dirty: 72428 kB
Mapped: 70192 kB
DirectMap4k: 100352 kB
所以这台机器需要一些时间才能启动失败,而相同的机器报告DirectMap4k:77824 kB并因此定期失败(它还必须处理构建更复杂的容器),但sysctl vm是相同的.
构建/启动docker容器的根本问题是抛出内存错误,问题是需要调整内核以使其稳定.
Docker版本
Client:
Version: 17.06.0-ce
API version: 1.30
Go version: go1.8.3
Git commit: 02c1d87
Built: Fri Jun 23 21:20:36 2017
OS/Arch: linux/amd64
Server:
Version: 17.06.0-ce
API version: 1.30 (minimum version 1.12)
Go version: go1.8.3
Git commit: 02c1d87
Built: Fri Jun 23 21:21:56 2017
OS/Arch: linux/amd64
Experimental: false
内核3.10.0-327.10.1.el7.x86_64
sysctl vm
vm.admin_reserve_kbytes = 8192
vm.block_dump = 0
vm.dirty_background_bytes = 0
vm.dirty_background_ratio = 10
vm.dirty_bytes = 0
vm.dirty_expire_centisecs = 3000
vm.dirty_ratio = 30
vm.dirty_writeback_centisecs = 500
vm.drop_caches = 1
vm.extfrag_threshold = 500
vm.hugepages_treat_as_movable = 0
vm.hugetlb_shm_group = 0
vm.laptop_mode = 0
vm.legacy_va_layout = 0
vm.lowmem_reserve_ratio = 256 256 32
vm.max_map_count = 65530
vm.memory_failure_early_kill = 0
vm.memory_failure_recovery = 1
vm.min_free_kbytes = 67584
vm.min_slab_ratio = 5
vm.min_unmapped_ratio = 1
vm.mmap_min_addr = 4096
vm.nr_hugepages = 0
vm.nr_hugepages_mempolicy = 0
vm.nr_overcommit_hugepages = 0
vm.nr_pdflush_threads = 0
vm.numa_zonelist_order = default
vm.oom_dump_tasks = 1
vm.oom_kill_allocating_task = 0
vm.overcommit_kbytes = 0
vm.overcommit_memory = 0
vm.overcommit_ratio = 50
vm.page-cluster = 3
vm.panic_on_oom = 0
vm.percpu_pagelist_fraction = 0
vm.stat_interval = 1
vm.swappiness = 30
vm.user_reserve_kbytes = 108990
vm.vfs_cache_pressure = 100
vm.zone_reclaim_mode = 0
本文由 投稿者 创作,文章地址:https://blog.isoyu.com/archives/rhel-dockerwufafenpeineicun-xuninei.html
采用知识共享署名4.0 国际许可协议进行许可。除注明转载/出处外,均为本站原创或翻译,转载前请务必署名。最后编辑时间为:9 月 12, 2019 at 06:51 下午