SaveText.Ru

Без имени
  1. [email protected]:/var/log/ceph# ceph health detail
  2. HEALTH_WARN 89 pgs backfill; 48 pgs degraded; 10 pgs down; 12 pgs incomplete; 2 pgs recovering; 15 pgs recovery_wait; 48 pgs stuck degraded; 12 pgs stuck inactive; 118 pgs stuck unclean; 35 pgs stuck undersized; 35 pgs undersized; 137 requests are blocked > 32 sec; 3 osds have slow requests; recovery 186086/3319779 objects degraded (5.605%); recovery 1092045/3319779 objects misplaced (32.895%); recovery 5/1426437 unfound (0.000%)
  3. pg 2.4d is stuck inactive since forever, current state incomplete, last acting [1,11]
  4. pg 19.d is stuck inactive since forever, current state down+incomplete, last acting [3,1]
  5. pg 12.10 is stuck inactive since forever, current state down+incomplete, last acting [5,10]
  6. pg 0.1c is stuck inactive since forever, current state down+incomplete, last acting [5,10]
  7. pg 16.c is stuck inactive since forever, current state down+incomplete, last acting [5,10]
  8. pg 1.1b is stuck inactive since forever, current state down+incomplete, last acting [5,10]
  9. pg 12.3c is stuck inactive since forever, current state down+incomplete, last acting [2,15]
  10. pg 2.1a is stuck inactive since forever, current state down+incomplete, last acting [5,10]
  11. pg 19.4e is stuck inactive since forever, current state incomplete, last acting [9,2]
  12. pg 12.51 is stuck inactive since forever, current state down+incomplete, last acting [3,10]
  13. pg 2.46 is stuck inactive since forever, current state down+incomplete, last acting [2,15]
  14. pg 16.38 is stuck inactive since forever, current state down+incomplete, last acting [2,15]
  15. pg 2.2f is stuck unclean for 244831.589098, current state active+undersized+degraded+remapped+wait_backfill, last acting [0]
  16. pg 16.3d is stuck unclean for 185906.560374, current state active+recovery_wait+undersized+degraded+remapped, last acting [9]
  17. pg 16.3b is stuck unclean for 145489.259543, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  18. pg 2.28 is stuck unclean for 166271.887136, current state active+remapped+wait_backfill, last acting [3,7]
  19. pg 16.3a is stuck unclean for 185904.609466, current state active+remapped+wait_backfill, last acting [5,9]
  20. pg 16.39 is stuck unclean for 48461.586169, current state active+remapped+wait_backfill, last acting [3,14]
  21. pg 16.38 is stuck unclean since forever, current state down+incomplete, last acting [2,15]
  22. pg 2.26 is stuck unclean for 156934.117083, current state active+remapped+wait_backfill, last acting [7,3]
  23. pg 16.34 is stuck unclean for 139904.812549, current state active+remapped+wait_backfill, last acting [0,3]
  24. pg 2.21 is stuck unclean for 195201.223480, current state active+remapped+wait_backfill, last acting [5,10]
  25. pg 16.32 is stuck unclean for 183908.520062, current state active+recovering+undersized+degraded+remapped, last acting [10]
  26. pg 16.31 is stuck unclean for 181425.997562, current state active+recovering+undersized+degraded+remapped, last acting [0]
  27. pg 2.1c is stuck unclean for 166440.846907, current state active+remapped+wait_backfill, last acting [8,2]
  28. pg 19.d is stuck unclean since forever, current state down+incomplete, last acting [3,1]
  29. pg 16.c is stuck unclean since forever, current state down+incomplete, last acting [5,10]
  30. pg 0.1c is stuck unclean since forever, current state down+incomplete, last acting [5,10]
  31. pg 12.10 is stuck unclean since forever, current state down+incomplete, last acting [5,10]
  32. pg 16.b is stuck unclean for 176090.191993, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  33. pg 2.19 is stuck unclean for 176227.650758, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  34. pg 1.1b is stuck unclean since forever, current state down+incomplete, last acting [5,10]
  35. pg 2.1a is stuck unclean since forever, current state down+incomplete, last acting [5,10]
  36. pg 16.6 is stuck unclean for 186171.285937, current state active+remapped+wait_backfill, last acting [3,15]
  37. pg 2.14 is stuck unclean for 62024.752107, current state active+remapped+wait_backfill, last acting [0,11]
  38. pg 16.5 is stuck unclean for 48437.682312, current state active+remapped+wait_backfill, last acting [0,15]
  39. pg 16.3 is stuck unclean for 176062.589019, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  40. pg 2.11 is stuck unclean for 176227.906414, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  41. pg 2.13 is stuck unclean for 176067.554830, current state active+remapped+wait_backfill, last acting [0,15]
  42. pg 16.1e is stuck unclean for 48310.922147, current state active+remapped+wait_backfill, last acting [10,5]
  43. pg 2.9 is stuck unclean for 184255.248598, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  44. pg 16.1a is stuck unclean for 48438.266099, current state active+remapped+wait_backfill, last acting [10,3]
  45. pg 16.18 is stuck unclean for 55203.601222, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  46. pg 2.1 is stuck unclean for 180689.195301, current state active+remapped+wait_backfill, last acting [8,14]
  47. pg 2.3 is stuck unclean for 176227.725487, current state active+remapped+wait_backfill, last acting [7,15]
  48. pg 16.10 is stuck unclean for 47457.699840, current state active+recovery_wait+degraded, last acting [10,11]
  49. pg 2.2 is stuck unclean for 244516.510305, current state active+remapped+wait_backfill, last acting [7,3]
  50. pg 16.88 is stuck unclean for 70394.710381, current state active+remapped+wait_backfill, last acting [0,3]
  51. pg 2.94 is stuck unclean for 180516.668178, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  52. pg 16.86 is stuck unclean for 180674.869366, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  53. pg 16.85 is stuck unclean for 140934.778411, current state active+remapped+wait_backfill, last acting [0,5]
  54. pg 2.93 is stuck unclean for 180570.866265, current state active+remapped+wait_backfill, last acting [0,5]
  55. pg 16.81 is stuck unclean for 52830.542654, current state active+recovery_wait+degraded+remapped, last acting [0,2]
  56. pg 2.8c is stuck unclean for 180114.626613, current state active+remapped+wait_backfill, last acting [10,5]
  57. pg 2.8f is stuck unclean for 180433.258529, current state active+remapped+wait_backfill, last acting [0,2]
  58. pg 2.85 is stuck unclean for 180499.082177, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  59. pg 2.86 is stuck unclean for 244701.927654, current state active+remapped+wait_backfill, last acting [1,9]
  60. pg 2.7c is stuck unclean for 180436.708246, current state active+remapped+wait_backfill, last acting [9,5]
  61. pg 16.6e is stuck unclean for 48437.071787, current state active+recovery_wait+degraded+remapped, last acting [9,5]
  62. pg 2.7e is stuck unclean for 48438.263698, current state active+remapped+wait_backfill, last acting [8,10]
  63. pg 2.79 is stuck unclean for 61800.766607, current state active+remapped+wait_backfill, last acting [7,10]
  64. pg 16.6b is stuck unclean for 65294.636231, current state active+recovery_wait+degraded+remapped, last acting [7,10]
  65. pg 2.78 is stuck unclean for 186239.997575, current state active+remapped+wait_backfill, last acting [9,3]
  66. pg 16.6a is stuck unclean for 185909.265894, current state active+recovery_wait+degraded+remapped, last acting [9,3]
  67. pg 16.65 is stuck unclean for 176167.965626, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  68. pg 2.71 is stuck unclean for 69797.267827, current state active+remapped+wait_backfill, last acting [8,3]
  69. pg 16.63 is stuck unclean for 68863.944001, current state active+recovery_wait+degraded+remapped, last acting [8,3]
  70. pg 2.73 is stuck unclean for 176227.804502, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  71. pg 16.61 is stuck unclean for 48438.377008, current state active+remapped+wait_backfill, last acting [11,2]
  72. pg 16.7e is stuck unclean for 47841.598081, current state active+remapped+wait_backfill, last acting [10,5]
  73. pg 2.6f is stuck unclean for 176061.844411, current state active+remapped+wait_backfill, last acting [11,2]
  74. pg 2.6b is stuck unclean for 180138.668445, current state active+remapped+wait_backfill, last acting [10,2]
  75. pg 2.6a is stuck unclean for 144964.286175, current state active+remapped+wait_backfill, last acting [3,1]
  76. pg 16.77 is stuck unclean for 176114.105923, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  77. pg 16.75 is stuck unclean for 173547.797952, current state active+remapped+wait_backfill, last acting [9,12]
  78. pg 2.67 is stuck unclean for 176061.757148, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  79. pg 16.70 is stuck unclean for 48437.034889, current state active+remapped+wait_backfill, last acting [8,10]
  80. pg 2.62 is stuck unclean for 244486.308228, current state active+undersized+degraded+remapped+wait_backfill, last acting [6]
  81. pg 16.4f is stuck unclean for 45278.958584, current state active+recovery_wait+degraded, last acting [7,2]
  82. pg 16.4e is stuck unclean for 70421.968837, current state active+recovery_wait+degraded+remapped, last acting [8,3]
  83. pg 2.5c is stuck unclean for 48843.826324, current state active+remapped+wait_backfill, last acting [8,3]
  84. pg 12.51 is stuck unclean since forever, current state down+incomplete, last acting [3,10]
  85. pg 19.4e is stuck unclean since forever, current state incomplete, last acting [9,2]
  86. pg 2.5f is stuck unclean for 166180.602940, current state active+remapped+wait_backfill, last acting [0,11]
  87. pg 16.4c is stuck unclean for 48438.392510, current state active+remapped+wait_backfill, last acting [1,12]
  88. pg 2.5e is stuck unclean for 244522.704464, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  89. pg 16.4b is stuck unclean for 184244.044816, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  90. pg 2.59 is stuck unclean for 220241.740091, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  91. pg 2.5a is stuck unclean for 180712.835632, current state active+remapped+wait_backfill, last acting [1,12]
  92. pg 16.46 is stuck unclean for 185890.491953, current state active+remapped+wait_backfill, last acting [9,3]
  93. pg 2.54 is stuck unclean for 48438.342533, current state active+remapped+wait_backfill, last acting [7,3]
  94. pg 16.45 is stuck unclean for 185885.530523, current state active+recovery_wait+degraded+remapped, last acting [7,3]
  95. pg 16.44 is stuck unclean for 184275.072806, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  96. pg 16.43 is stuck unclean for 179415.655948, current state active+recovery_wait+undersized+degraded+remapped, last acting [1]
  97. pg 2.53 is stuck unclean for 221847.162741, current state active+remapped+wait_backfill, last acting [7,3]
  98. pg 16.41 is stuck unclean for 184342.628275, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  99. pg 2.52 is stuck unclean for 244424.314907, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  100. pg 2.4d is stuck unclean since forever, current state incomplete, last acting [1,11]
  101. pg 2.4c is stuck unclean for 48438.279533, current state active+remapped+wait_backfill, last acting [8,10]
  102. pg 16.5d is stuck unclean for 48436.977170, current state active+remapped+wait_backfill, last acting [10,2]
  103. pg 16.5c is stuck unclean for 55536.780530, current state active+remapped+wait_backfill, last acting [3,1]
  104. pg 2.49 is stuck unclean for 52871.302321, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  105. pg 16.5b is stuck unclean for 45280.551646, current state active+recovery_wait+degraded, last acting [1,6]
  106. pg 16.5a is stuck unclean for 173558.861647, current state active+remapped+wait_backfill, last acting [7,15]
  107. pg 2.4b is stuck unclean for 185903.285665, current state active+remapped+wait_backfill, last acting [9,5]
  108. pg 16.59 is stuck unclean for 176422.625331, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  109. pg 2.4a is stuck unclean for 61800.792851, current state active+remapped+wait_backfill, last acting [10,7]
  110. pg 2.47 is stuck unclean for 173453.301133, current state active+remapped+wait_backfill, last acting [3,14]
  111. pg 2.46 is stuck unclean since forever, current state down+incomplete, last acting [2,15]
  112. pg 16.54 is stuck unclean for 45285.804171, current state active+recovery_wait+degraded, last acting [11,2]
  113. pg 16.51 is stuck unclean for 137322.606075, current state active+remapped+wait_backfill, last acting [0,11]
  114. pg 16.50 is stuck unclean for 185883.947565, current state active+recovery_wait+degraded+remapped, last acting [8,2]
  115. pg 2.3c is stuck unclean for 185932.718914, current state active+remapped+wait_backfill, last acting [9,1]
  116. pg 16.2e is stuck unclean for 185927.695435, current state active+remapped+wait_backfill, last acting [1,9]
  117. pg 2.3f is stuck unclean for 180169.080578, current state active+undersized+degraded+remapped+wait_backfill, last acting [0]
  118. pg 16.2d is stuck unclean for 184238.081324, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  119. pg 2.39 is stuck unclean for 69456.532177, current state active+remapped+wait_backfill, last acting [8,0]
  120. pg 16.2b is stuck unclean for 180557.794320, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  121. pg 16.2a is stuck unclean for 61800.678553, current state active+remapped+wait_backfill, last acting [8,0]
  122. pg 2.3b is stuck unclean for 220139.210351, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  123. pg 2.35 is stuck unclean for 166351.262301, current state active+remapped+wait_backfill, last acting [3,8]
  124. pg 2.34 is stuck unclean for 179994.508676, current state active+remapped+wait_backfill, last acting [7,3]
  125. pg 16.26 is stuck unclean for 48461.667961, current state active+recovery_wait+degraded+remapped, last acting [7,3]
  126. pg 2.30 is stuck unclean for 176227.860151, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  127. pg 16.22 is stuck unclean for 176095.999205, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  128. pg 16.21 is stuck unclean for 185932.051202, current state active+undersized+degraded+remapped+wait_backfill, last acting [0]
  129. pg 12.3c is stuck unclean since forever, current state down+incomplete, last acting [2,15]
  130. pg 16.3f is stuck unclean for 195022.936734, current state active+remapped+wait_backfill, last acting [1,12]
  131. pg 2.2c is stuck unclean for 180417.712715, current state active+remapped+wait_backfill, last acting [10,5]
  132. pg 16.3e is stuck unclean for 48437.078210, current state active+remapped+wait_backfill, last acting [8,10]
  133. pg 16.3d is stuck undersized for 45250.064802, current state active+recovery_wait+undersized+degraded+remapped, last acting [9]
  134. pg 2.2f is stuck undersized for 45258.785354, current state active+undersized+degraded+remapped+wait_backfill, last acting [0]
  135. pg 16.3b is stuck undersized for 45246.540843, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  136. pg 16.32 is stuck undersized for 45239.429627, current state active+recovering+undersized+degraded+remapped, last acting [10]
  137. pg 16.31 is stuck undersized for 45258.792493, current state active+recovering+undersized+degraded+remapped, last acting [0]
  138. pg 16.b is stuck undersized for 45246.609691, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  139. pg 2.19 is stuck undersized for 45246.620544, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  140. pg 16.3 is stuck undersized for 45250.012185, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  141. pg 2.11 is stuck undersized for 45250.019849, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  142. pg 2.9 is stuck undersized for 45251.470147, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  143. pg 16.18 is stuck undersized for 45239.486845, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  144. pg 16.86 is stuck undersized for 45249.742107, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  145. pg 2.94 is stuck undersized for 45249.740942, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  146. pg 2.85 is stuck undersized for 45249.760442, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  147. pg 16.65 is stuck undersized for 45249.853818, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  148. pg 2.73 is stuck undersized for 45249.863420, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  149. pg 16.77 is stuck undersized for 45249.742726, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  150. pg 2.67 is stuck undersized for 45246.622342, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  151. pg 2.62 is stuck undersized for 45239.343430, current state active+undersized+degraded+remapped+wait_backfill, last acting [6]
  152. pg 2.5e is stuck undersized for 45237.805718, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  153. pg 16.4b is stuck undersized for 45238.399333, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  154. pg 2.59 is stuck undersized for 45238.409253, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  155. pg 16.44 is stuck undersized for 45249.751698, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  156. pg 16.43 is stuck undersized for 45258.711476, current state active+recovery_wait+undersized+degraded+remapped, last acting [1]
  157. pg 16.41 is stuck undersized for 45239.508277, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  158. pg 2.52 is stuck undersized for 45249.760582, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  159. pg 2.49 is stuck undersized for 45246.545566, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  160. pg 16.59 is stuck undersized for 45246.627318, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  161. pg 16.2d is stuck undersized for 45238.424841, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  162. pg 2.3f is stuck undersized for 45258.749698, current state active+undersized+degraded+remapped+wait_backfill, last acting [0]
  163. pg 16.2b is stuck undersized for 45239.518763, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  164. pg 2.3b is stuck undersized for 45238.431500, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  165. pg 16.22 is stuck undersized for 45251.445982, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  166. pg 2.30 is stuck undersized for 45251.443936, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  167. pg 16.21 is stuck undersized for 45258.777967, current state active+undersized+degraded+remapped+wait_backfill, last acting [0]
  168. pg 16.3d is stuck degraded for 45250.065078, current state active+recovery_wait+undersized+degraded+remapped, last acting [9]
  169. pg 2.2f is stuck degraded for 45258.785630, current state active+undersized+degraded+remapped+wait_backfill, last acting [0]
  170. pg 16.3b is stuck degraded for 45246.541119, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  171. pg 16.32 is stuck degraded for 45239.429903, current state active+recovering+undersized+degraded+remapped, last acting [10]
  172. pg 16.31 is stuck degraded for 45258.792770, current state active+recovering+undersized+degraded+remapped, last acting [0]
  173. pg 16.b is stuck degraded for 45246.609967, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  174. pg 2.19 is stuck degraded for 45246.620820, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  175. pg 16.3 is stuck degraded for 45250.012461, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  176. pg 2.11 is stuck degraded for 45250.020125, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  177. pg 2.9 is stuck degraded for 45251.470423, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  178. pg 16.18 is stuck degraded for 45239.487121, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  179. pg 16.10 is stuck degraded for 45239.164720, current state active+recovery_wait+degraded, last acting [10,11]
  180. pg 16.86 is stuck degraded for 45249.742389, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  181. pg 2.94 is stuck degraded for 45249.741224, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  182. pg 16.81 is stuck degraded for 45240.507627, current state active+recovery_wait+degraded+remapped, last acting [0,2]
  183. pg 2.85 is stuck degraded for 45249.760730, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  184. pg 16.6e is stuck degraded for 45237.816714, current state active+recovery_wait+degraded+remapped, last acting [9,5]
  185. pg 16.6b is stuck degraded for 45240.372333, current state active+recovery_wait+degraded+remapped, last acting [7,10]
  186. pg 16.6a is stuck degraded for 45239.119347, current state active+recovery_wait+degraded+remapped, last acting [9,3]
  187. pg 16.65 is stuck degraded for 45249.854124, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  188. pg 16.63 is stuck degraded for 45246.394690, current state active+recovery_wait+degraded+remapped, last acting [8,3]
  189. pg 2.73 is stuck degraded for 45249.863733, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  190. pg 16.77 is stuck degraded for 45249.743038, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  191. pg 2.67 is stuck degraded for 45246.622654, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  192. pg 2.62 is stuck degraded for 45239.343743, current state active+undersized+degraded+remapped+wait_backfill, last acting [6]
  193. pg 16.4f is stuck degraded for 45252.396042, current state active+recovery_wait+degraded, last acting [7,2]
  194. pg 16.4e is stuck degraded for 45246.459587, current state active+recovery_wait+degraded+remapped, last acting [8,3]
  195. pg 2.5e is stuck degraded for 45237.806043, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  196. pg 16.4b is stuck degraded for 45238.399658, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  197. pg 2.59 is stuck degraded for 45238.409578, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  198. pg 16.45 is stuck degraded for 45251.206581, current state active+recovery_wait+degraded+remapped, last acting [7,3]
  199. pg 16.44 is stuck degraded for 45249.752029, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  200. pg 16.43 is stuck degraded for 45258.711807, current state active+recovery_wait+undersized+degraded+remapped, last acting [1]
  201. pg 16.41 is stuck degraded for 45239.508608, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  202. pg 2.52 is stuck degraded for 45249.760913, current state active+undersized+degraded+remapped+wait_backfill, last acting [9]
  203. pg 16.5b is stuck degraded for 45254.719517, current state active+recovery_wait+degraded, last acting [1,6]
  204. pg 2.49 is stuck degraded for 45246.545903, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  205. pg 16.59 is stuck degraded for 45246.627654, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  206. pg 16.54 is stuck degraded for 45239.030398, current state active+recovery_wait+degraded, last acting [11,2]
  207. pg 16.50 is stuck degraded for 45237.770003, current state active+recovery_wait+degraded+remapped, last acting [8,2]
  208. pg 16.2d is stuck degraded for 45238.425189, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  209. pg 2.3f is stuck degraded for 45258.750046, current state active+undersized+degraded+remapped+wait_backfill, last acting [0]
  210. pg 16.2b is stuck degraded for 45239.519110, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  211. pg 2.3b is stuck degraded for 45238.431848, current state active+undersized+degraded+remapped+wait_backfill, last acting [8]
  212. pg 16.26 is stuck degraded for 45252.375929, current state active+recovery_wait+degraded+remapped, last acting [7,3]
  213. pg 16.22 is stuck degraded for 45251.446336, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  214. pg 2.30 is stuck degraded for 45251.444289, current state active+undersized+degraded+remapped+wait_backfill, last acting [7]
  215. pg 16.21 is stuck degraded for 45258.778321, current state active+undersized+degraded+remapped+wait_backfill, last acting [0]
  216. pg 16.88 is active+remapped+wait_backfill, acting [0,3]
  217. pg 2.94 is active+undersized+degraded+remapped+wait_backfill, acting [9]
  218. pg 16.86 is active+undersized+degraded+remapped+wait_backfill, acting [9]
  219. pg 16.85 is active+remapped+wait_backfill, acting [0,5]
  220. pg 2.93 is active+remapped+wait_backfill, acting [0,5]
  221. pg 16.81 is active+recovery_wait+degraded+remapped, acting [0,2]
  222. pg 2.8c is active+remapped+wait_backfill, acting [10,5]
  223. pg 2.8f is active+remapped+wait_backfill, acting [0,2]
  224. pg 2.85 is active+undersized+degraded+remapped+wait_backfill, acting [9]
  225. pg 2.86 is active+remapped+wait_backfill, acting [1,9]
  226. pg 2.7c is active+remapped+wait_backfill, acting [9,5]
  227. pg 16.6e is active+recovery_wait+degraded+remapped, acting [9,5]
  228. pg 2.7e is active+remapped+wait_backfill, acting [8,10]
  229. pg 2.79 is active+remapped+wait_backfill, acting [7,10]
  230. pg 16.6b is active+recovery_wait+degraded+remapped, acting [7,10]
  231. pg 2.78 is active+remapped+wait_backfill, acting [9,3]
  232. pg 16.6a is active+recovery_wait+degraded+remapped, acting [9,3]
  233. pg 16.65 is active+undersized+degraded+remapped+wait_backfill, acting [9]
  234. pg 2.71 is active+remapped+wait_backfill, acting [8,3]
  235. pg 16.63 is active+recovery_wait+degraded+remapped, acting [8,3]
  236. pg 2.73 is active+undersized+degraded+remapped+wait_backfill, acting [9]
  237. pg 16.61 is active+remapped+wait_backfill, acting [11,2]
  238. pg 16.7e is active+remapped+wait_backfill, acting [10,5]
  239. pg 2.6f is active+remapped+wait_backfill, acting [11,2]
  240. pg 2.6b is active+remapped+wait_backfill, acting [10,2]
  241. pg 2.6a is active+remapped+wait_backfill, acting [3,1]
  242. pg 16.77 is active+undersized+degraded+remapped+wait_backfill, acting [9]
  243. pg 2.67 is active+undersized+degraded+remapped+wait_backfill, acting [8]
  244. pg 16.75 is active+remapped+wait_backfill, acting [9,12]
  245. pg 2.62 is active+undersized+degraded+remapped+wait_backfill, acting [6]
  246. pg 16.70 is active+remapped+wait_backfill, acting [8,10]
  247. pg 16.4f is active+recovery_wait+degraded, acting [7,2]
  248. pg 2.5c is active+remapped+wait_backfill, acting [8,3]
  249. pg 16.4e is active+recovery_wait+degraded+remapped, acting [8,3]
  250. pg 2.5f is active+remapped+wait_backfill, acting [0,11]
  251. pg 19.4e is incomplete, acting [9,2]
  252. pg 12.51 is down+incomplete, acting [3,10]
  253. pg 2.5e is active+undersized+degraded+remapped+wait_backfill, acting [8]
  254. pg 16.4c is active+remapped+wait_backfill, acting [1,12]
  255. pg 2.59 is active+undersized+degraded+remapped+wait_backfill, acting [8]
  256. pg 16.4b is active+undersized+degraded+remapped+wait_backfill, acting [8]
  257. pg 2.5a is active+remapped+wait_backfill, acting [1,12]
  258. pg 2.54 is active+remapped+wait_backfill, acting [7,3]
  259. pg 16.46 is active+remapped+wait_backfill, acting [9,3]
  260. pg 16.45 is active+recovery_wait+degraded+remapped, acting [7,3]
  261. pg 16.44 is active+undersized+degraded+remapped+wait_backfill, acting [9]
  262. pg 16.43 is active+recovery_wait+undersized+degraded+remapped, acting [1], 1 unfound
  263. pg 2.53 is active+remapped+wait_backfill, acting [7,3]
  264. pg 16.41 is active+undersized+degraded+remapped+wait_backfill, acting [8]
  265. pg 2.52 is active+undersized+degraded+remapped+wait_backfill, acting [9]
  266. pg 2.4d is incomplete, acting [1,11]
  267. pg 2.4c is active+remapped+wait_backfill, acting [8,10]
  268. pg 16.5d is active+remapped+wait_backfill, acting [10,2]
  269. pg 16.5c is active+remapped+wait_backfill, acting [3,1]
  270. pg 2.49 is active+undersized+degraded+remapped+wait_backfill, acting [8]
  271. pg 16.5b is active+recovery_wait+degraded, acting [1,6]
  272. pg 16.5a is active+remapped+wait_backfill, acting [7,15]
  273. pg 2.4b is active+remapped+wait_backfill, acting [9,5]
  274. pg 16.59 is active+undersized+degraded+remapped+wait_backfill, acting [8]
  275. pg 2.4a is active+remapped+wait_backfill, acting [10,7]
  276. pg 2.47 is active+remapped+wait_backfill, acting [3,14]
  277. pg 2.46 is down+incomplete, acting [2,15]
  278. pg 16.54 is active+recovery_wait+degraded, acting [11,2]
  279. pg 16.51 is active+remapped+wait_backfill, acting [0,11]
  280. pg 16.50 is active+recovery_wait+degraded+remapped, acting [8,2]
  281. pg 2.3c is active+remapped+wait_backfill, acting [9,1]
  282. pg 16.2e is active+remapped+wait_backfill, acting [1,9]
  283. pg 2.3f is active+undersized+degraded+remapped+wait_backfill, acting [0]
  284. pg 16.2d is active+undersized+degraded+remapped+wait_backfill, acting [8]
  285. pg 2.39 is active+remapped+wait_backfill, acting [8,0]
  286. pg 16.2b is active+undersized+degraded+remapped+wait_backfill, acting [8]
  287. pg 16.2a is active+remapped+wait_backfill, acting [8,0]
  288. pg 2.3b is active+undersized+degraded+remapped+wait_backfill, acting [8]
  289. pg 2.35 is active+remapped+wait_backfill, acting [3,8]
  290. pg 2.34 is active+remapped+wait_backfill, acting [7,3]
  291. pg 16.26 is active+recovery_wait+degraded+remapped, acting [7,3]
  292. pg 2.30 is active+undersized+degraded+remapped+wait_backfill, acting [7]
  293. pg 16.22 is active+undersized+degraded+remapped+wait_backfill, acting [7]
  294. pg 16.21 is active+undersized+degraded+remapped+wait_backfill, acting [0]
  295. pg 12.3c is down+incomplete, acting [2,15]
  296. pg 16.3f is active+remapped+wait_backfill, acting [1,12]
  297. pg 2.2c is active+remapped+wait_backfill, acting [10,5]
  298. pg 16.3e is active+remapped+wait_backfill, acting [8,10]
  299. pg 2.2f is active+undersized+degraded+remapped+wait_backfill, acting [0]
  300. pg 16.3d is active+recovery_wait+undersized+degraded+remapped, acting [9]
  301. pg 16.3b is active+undersized+degraded+remapped+wait_backfill, acting [8]
  302. pg 2.28 is active+remapped+wait_backfill, acting [3,7]
  303. pg 16.3a is active+remapped+wait_backfill, acting [5,9]
  304. pg 16.39 is active+remapped+wait_backfill, acting [3,14]
  305. pg 16.38 is down+incomplete, acting [2,15]
  306. pg 2.26 is active+remapped+wait_backfill, acting [7,3]
  307. pg 16.34 is active+remapped+wait_backfill, acting [0,3]
  308. pg 2.21 is active+remapped+wait_backfill, acting [5,10]
  309. pg 16.32 is active+recovering+undersized+degraded+remapped, acting [10], 2 unfound
  310. pg 16.31 is active+recovering+undersized+degraded+remapped, acting [0], 2 unfound
  311. pg 2.1c is active+remapped+wait_backfill, acting [8,2]
  312. pg 19.d is down+incomplete, acting [3,1]
  313. pg 12.10 is down+incomplete, acting [5,10]
  314. pg 0.1c is down+incomplete, acting [5,10]
  315. pg 16.c is down+incomplete, acting [5,10]
  316. pg 2.19 is active+undersized+degraded+remapped+wait_backfill, acting [8]
  317. pg 16.b is active+undersized+degraded+remapped+wait_backfill, acting [8]
  318. pg 1.1b is down+incomplete, acting [5,10]
  319. pg 2.1a is down+incomplete, acting [5,10]
  320. pg 2.14 is active+remapped+wait_backfill, acting [0,11]
  321. pg 16.6 is active+remapped+wait_backfill, acting [3,15]
  322. pg 16.5 is active+remapped+wait_backfill, acting [0,15]
  323. pg 2.11 is active+undersized+degraded+remapped+wait_backfill, acting [9]
  324. pg 16.3 is active+undersized+degraded+remapped+wait_backfill, acting [9]
  325. pg 2.13 is active+remapped+wait_backfill, acting [0,15]
  326. pg 16.1e is active+remapped+wait_backfill, acting [10,5]
  327. pg 2.9 is active+undersized+degraded+remapped+wait_backfill, acting [7]
  328. pg 16.1a is active+remapped+wait_backfill, acting [10,3]
  329. pg 16.18 is active+undersized+degraded+remapped+wait_backfill, acting [7]
  330. pg 2.1 is active+remapped+wait_backfill, acting [8,14]
  331. pg 2.3 is active+remapped+wait_backfill, acting [7,15]
  332. pg 2.2 is active+remapped+wait_backfill, acting [7,3]
  333. pg 16.10 is active+recovery_wait+degraded, acting [10,11]
  334. 101 ops are blocked > 67108.9 sec
  335. 5 ops are blocked > 33554.4 sec
  336. 23 ops are blocked > 4194.3 sec
  337. 8 ops are blocked > 2097.15 sec
  338. 100 ops are blocked > 67108.9 sec on osd.2
  339. 5 ops are blocked > 33554.4 sec on osd.5
  340. 23 ops are blocked > 4194.3 sec on osd.5
  341. 8 ops are blocked > 2097.15 sec on osd.5
  342. 1 ops are blocked > 67108.9 sec on osd.10
  343. 3 osds have slow requests
  344. recovery 186086/3319779 objects degraded (5.605%)
  345. recovery 1092045/3319779 objects misplaced (32.895%)
  346. recovery 5/1426437 unfound (0.000%)
  347. [email protected]:/var/log/ceph#

Share with your friends:

Print