There are a lot of spurious exceptions in the kubernetes kubelet logs
like:
E1018 21:03:09.616581 22780 fs.go:332] Stat fs failed. Error: no such
file or directory
Since we know that calling syscall.Statfs will just fail when the path
does not exist, we should just skip making the call.
NOTE: fixing 2017->2018 problems in build by running `./build/jenkins_e2e.sh`
As these can otherwise block indefinitely due to docker issues.
This is to fix https://github.com/kubernetes/kubernetes/issues/53207,
where kubelet relies on cadvisor for gathering docker information as
part of its periodic node status update.
This commit includes changes to integrate containerd
runtime to cadvisor to collect container stats
Signed-off-by: abhi <abhi@docker.com>
Test cases and minor changes
This commit include test cases and minor fixes
for the same
Signed-off-by: abhi <abhi@docker.com>
This commit includes godep change to update the grpc version
and also updates rkt version to v1.25.0.
A minor change has been made in the client based on how rkt
client is used in kubernetes/kubernetes.
Signed-off-by: abhi <abhi@docker.com>
When cAdvisor starts up, it would read the `vendor` files in
`/sys/bus/pci/devices/*` to see if any NVIDIA devices (vendor ID: 0x10de) are
attached to the node. If no NVIDIA devices are found, this code path would
become dormant for the rest of cAdvisor lifetime. If NVIDIA devices are found,
we would start a goroutine that would check for the presence of NVML by trying
to dynamically load it at regular intervals. We need to do this regular
checking instead of doing it just once because it may happen that cAdvisor is
started before the NVIDIA drivers and NVML are installed. Once the NVML
dynamic loading succeeds, we would use NVML’s query methods to find out how
many devices exist on the node and create a map from their minor numbers to
their handles and cache that map. The goroutine would exit at this point.
If we detected the presence of NVML in the previous step, whenever a new
container is detected by cAdvisor, cAdvisor would read the `devices.list` file
from the container's devices cgroup. The `devices.list` file lists the
major:minor number of all the devices that the container is allowed to access.
If we find any device with major number 195 (which is the major number assigned
to NVIDIA devices), we would cache the list of corresponding minor numbers for
that container.
During every housekeeping operation, in addition to collecting all the existing
metrics, we will use the cached NVIDIA device minor numbers and the map from
minor numbers to device handles to get metrics for GPU devices attached to the
container.
We can't build a static binary because that would require bundling the
closed source NVML library in cAdvisor.
Instead, gonvml uses dlopen to dynamically load NVML.
