| NGI_FRANCE |
IN2P3-CC-T2 |
TIER2 |
2021-03-16 04:00 |
2021-03-16 20:00 |
OUTAGE
|
SCHEDULED
|
Scheduled site maintenance. More details during the month before the outage |
|
|
| NGI_RO |
RO-11-NIPNE |
TIER2 |
2021-01-12 10:23 |
2021-02-08 09:00 |
OUTAGE
|
SCHEDULED
|
Extended downtime for site relocation and software upgrades. |
|
|
| NGI_IT |
INFN-LNL-2 |
TIER2 |
2021-01-26 08:00 |
2021-01-26 12:38 |
OUTAGE
|
SCHEDULED
|
Storage downtime for SE software upgrade (dCache) |
|
|
| NGI_UK |
UKI-SOUTHGRID-BHAM-HEP |
TIER2 |
2021-01-27 08:00 |
2021-01-27 14:00 |
OUTAGE
|
SCHEDULED
|
Reinstall of epgperf onto new hardware |
|
|
| NGI_IT |
HEPHY-UIBK |
TIER2 |
2021-01-22 20:00 |
2021-01-29 20:00 |
OUTAGE
|
UNSCHEDULED
|
software problems |
|
|
| NGI_DE |
MPPMU |
TIER2 |
2021-01-27 07:00 |
2021-01-27 18:02 |
OUTAGE
|
SCHEDULED
|
infrastructure maintenance |
|
|
| NGI_CZ |
prague_cesnet_lcg2 |
TIER2 |
2021-01-25 23:00 |
2021-01-28 15:21 |
OUTAGE
|
UNSCHEDULED
|
voms2 wrong certificate |
|
|
| NGI_IBERGRID |
IFIC-LCG2 |
TIER2 |
2021-01-29 06:40 |
2021-01-29 07:30 |
WARNING
|
SCHEDULED
|
Campus network intervention |
|
|
| NGI_UK |
UKI-LT2-QMUL |
TIER2 |
2021-01-28 12:30 |
2021-01-28 15:00 |
OUTAGE
|
UNSCHEDULED
|
network instabilities, university has cut our link in effort to understand network problems around the university |
|
|
| NGI_UK |
UKI-LT2-QMUL |
TIER2 |
2021-01-28 15:00 |
2021-01-29 15:00 |
WARNING
|
UNSCHEDULED
|
network instabilities at the university. main site link is down |
|
|
| AsiaPacific |
INDIACMS-TIFR |
TIER2 |
2021-01-27 18:30 |
2021-01-29 15:58 |
OUTAGE
|
UNSCHEDULED
|
storage upgrade: unknown BAD Credentials error |
|
|
| NGI_UK |
UKI-LT2-QMUL |
TIER2 |
2021-01-29 09:00 |
2021-01-29 16:00 |
OUTAGE
|
UNSCHEDULED
|
University has decided to keep us off the network until they have a proved stable connection |
|
|
| NGI_IT |
INFN-ROMA1-CMS |
TIER2 |
2021-01-30 12:00 |
2021-02-01 06:35 |
OUTAGE
|
SCHEDULED
|
migration of CE from LSF7 to LSF9 |
|
|
| NGI_IT |
HEPHY-UIBK |
TIER2 |
2021-01-29 20:00 |
2021-02-12 20:00 |
OUTAGE
|
UNSCHEDULED
|
software failure - docker internal error |
|
|
| NGI_UK |
UKI-NORTHGRID-MAN-HEP |
TIER2 |
2021-02-01 00:00 |
2021-02-01 13:17 |
OUTAGE
|
UNSCHEDULED
|
DNS server down |
|
|
| NGI_DE |
RWTH-Aachen |
TIER2 |
2021-02-03 08:00 |
2021-02-03 16:00 |
OUTAGE
|
SCHEDULED
|
several upgrades |
|
|
| NGI_RO |
RO-11-NIPNE |
TIER2 |
2021-02-08 09:00 |
2021-03-15 10:00 |
OUTAGE
|
SCHEDULED
|
Extended downtime for site relocation and OS upgrade and reconfiguration. |
|
|
| ROC_Canada |
CA-VICTORIA-WESTGRID-T2 |
TIER2 |
2021-02-28 08:00 |
2021-12-31 08:00 |
OUTAGE
|
SCHEDULED
|
decommission Cream CEs |
|
|
| NGI_UK |
UKI-SCOTGRID-ECDF |
TIER2 |
2021-02-01 13:00 |
2021-02-02 17:00 |
OUTAGE
|
UNSCHEDULED
|
Unexpected site issues due to GlusterFS which backs the local compute cluster having problems. |
|
|
| NGI_NDGF |
FI_HIP_T2 |
TIER2 |
2021-02-04 10:00 |
2021-02-04 11:00 |
OUTAGE
|
SCHEDULED
|
Certificate renewals |
|
|
| NGI_PL |
CYFRONET-LCG2 |
TIER2 |
2021-02-04 09:00 |
2021-02-06 14:00 |
OUTAGE
|
SCHEDULED
|
upgrade to Centos 7, enabling IPv6 |
|
|
| NGI_UK |
UKI-SCOTGRID-ECDF |
TIER2 |
2021-02-02 17:00 |
2021-02-03 17:00 |
OUTAGE
|
UNSCHEDULED
|
GPFS needs replacement hardware |
|
|
| NGI_NDGF |
SE-SNIC-T2 |
TIER2 |
2021-02-05 06:00 |
2021-02-05 09:44 |
OUTAGE
|
SCHEDULED
|
Network restructuring of LUNARC dCache pools |
|
|
| NGI_IT |
INFN-PISA |
TIER2 |
2021-02-12 22:50 |
2021-02-15 22:50 |
OUTAGE
|
SCHEDULED
|
Draining site in preparation of site maintenance |
|
|
| NGI_IT |
INFN-PISA |
TIER2 |
2021-02-15 22:50 |
2021-02-18 16:00 |
OUTAGE
|
SCHEDULED
|
Site shutdown due to storage maintenance |
|
|
| NGI_DE |
RWTH-Aachen |
TIER2 |
2021-02-03 14:00 |
2021-02-03 18:31 |
OUTAGE
|
UNSCHEDULED
|
xrootd problems |
|
|
| NGI_UK |
UKI-SCOTGRID-ECDF |
TIER2 |
2021-02-03 17:00 |
2021-02-04 13:00 |
OUTAGE
|
UNSCHEDULED
|
GPFS needs replacement hardware and contractor is taking a long time to install replacement |
|
|
| NGI_NDGF |
SE-SNIC-T2 |
TIER2 |
2021-02-03 11:00 |
2021-02-04 10:17 |
OUTAGE
|
UNSCHEDULED
|
All ATLAS and half of the Alice storage is unavailable. Storage pool outage for unknown reasons. |
|
|
| NGI_DE |
LRZ-LMU |
TIER2 |
2021-02-03 15:00 |
2021-02-04 16:59 |
OUTAGE
|
UNSCHEDULED
|
WNs down for mainenance |
|
|
| NGI_UK |
UKI-SOUTHGRID-BRIS-HEP |
TIER2 |
2021-02-15 09:00 |
2021-02-16 09:00 |
OUTAGE
|
SCHEDULED
|
Site Network Maintenance |
|
|
| NGI_UK |
UKI-SOUTHGRID-BRIS-HEP |
TIER2 |
2021-02-16 09:00 |
2021-02-18 16:00 |
WARNING
|
SCHEDULED
|
Site Network Maintenance (prudent in case it goes awry)(again) |
|
|
| NGI_FRANCE |
GRIF |
TIER2 |
2021-02-06 01:00 |
2021-02-28 01:00 |
OUTAGE
|
SCHEDULED
|
The node is under decomission |
|
|
| ROC_Canada |
CA-VICTORIA-WESTGRID-T2 |
TIER2 |
2021-02-04 22:00 |
2021-02-05 00:30 |
WARNING
|
UNSCHEDULED
|
brief network interruptions temporarily affecting access to dcache doors |
|
|
| NGI_IBERGRID |
USC-LCG2 |
TIER2 |
2021-02-21 00:00 |
2021-03-21 00:00 |
OUTAGE
|
SCHEDULED
|
ce01.igfae.usc.es will be decommissioned on the following weeks. This downtime initiates the final shutdown procedure for this machine. |
|
|
| NGI_UK |
UKI-NORTHGRID-LANCS-HEP |
TIER2 |
2021-02-04 23:00 |
2021-02-05 18:31 |
OUTAGE
|
UNSCHEDULED
|
Cluster Filesystem problems preventing job submission |
|
|
| NGI_DE |
MPPMU |
TIER2 |
2021-02-05 13:00 |
2021-02-05 15:29 |
OUTAGE
|
UNSCHEDULED
|
cooling system failure |
|
|
| NGI_UK |
UKI-LT2-Brunel |
TIER2 |
2021-02-07 13:00 |
2021-02-09 10:00 |
OUTAGE
|
SCHEDULED
|
Submission issues |
|
|
| NGI_UK |
UKI-SCOTGRID-ECDF |
TIER2 |
2021-02-07 21:15 |
2021-02-08 13:00 |
OUTAGE
|
UNSCHEDULED
|
Outage due to regional power problems |
|
|
| NGI_CH |
CSCS-LCG2 |
TIER2 |
2021-02-10 06:00 |
2021-02-10 17:43 |
OUTAGE
|
SCHEDULED
|
CSCS - Sitewide Maintenance |
|
|
| NGI_UK |
UKI-SCOTGRID-DURHAM |
TIER2 |
2021-02-13 09:00 |
2021-02-13 17:00 |
WARNING
|
SCHEDULED
|
WAN Link Router Maintenence. |
|
|
| AsiaPacific |
INDIACMS-TIFR |
TIER2 |
2021-02-09 06:30 |
2021-02-10 18:00 |
OUTAGE
|
UNSCHEDULED
|
DPM upgrade to latest version |
|
|
| NGI_UK |
UKI-SCOTGRID-ECDF |
TIER2 |
2021-02-08 13:00 |
2021-02-10 12:00 |
OUTAGE
|
UNSCHEDULED
|
Regional power failure compounded by a cooler failure at datacenter requires servers remain offline. |
|
|
| NGI_UK |
UKI-NORTHGRID-LANCS-HEP |
TIER2 |
2021-02-08 13:45 |
2021-02-08 14:31 |
OUTAGE
|
UNSCHEDULED
|
Emergency LRMS intervention has caused us to need to shut off job submission. |
|
|
| NGI_UK |
UKI-LT2-QMUL |
TIER2 |
2021-02-10 10:00 |
2021-02-10 16:00 |
WARNING
|
SCHEDULED
|
firmware updates for Lustre storage nodes |
|
|
| NGI_UK |
UKI-LT2-QMUL |
TIER2 |
2021-02-09 06:45 |
2021-02-09 09:30 |
OUTAGE
|
UNSCHEDULED
|
security update on network requires external storage access to be stopped |
|
|
| ROC_Canada |
CA-VICTORIA-WESTGRID-T2 |
TIER2 |
2021-02-08 19:30 |
2021-02-08 21:30 |
WARNING
|
UNSCHEDULED
|
short network interruptions due to configuration changes |
|
|
| NGI_IBERGRID |
CIEMAT-LCG2 |
TIER2 |
2021-02-11 06:00 |
2021-02-11 08:00 |
OUTAGE
|
SCHEDULED
|
Substitution of our main firewall system (hardware) |
|
|
| NGI_IBERGRID |
CIEMAT-LCG2 |
TIER2 |
2021-02-11 08:00 |
2021-02-11 10:00 |
OUTAGE
|
SCHEDULED
|
Substitution of our main firewall system (hardware) |
|
|
| AsiaPacific |
INDIACMS-TIFR |
TIER2 |
2021-02-10 18:00 |
2021-02-12 22:00 |
OUTAGE
|
UNSCHEDULED
|
DPM upgrade to latest version Extended Part2 |
|
|
| NGI_UK |
UKI-SCOTGRID-ECDF |
TIER2 |
2021-02-10 12:00 |
2021-02-12 17:00 |
OUTAGE
|
UNSCHEDULED
|
Hardware failure brought on by repeated power cycling at ACF requires manual intervention |
|
|
| NGI_IBERGRID |
CIEMAT-LCG2 |
TIER2 |
2021-02-11 10:38 |
2021-02-11 14:00 |
OUTAGE
|
UNSCHEDULED
|
General problems with institute network connectivity |
|
|
| NGI_SI |
SiGNET |
TIER2 |
2021-02-14 05:00 |
2021-02-14 07:30 |
WARNING
|
SCHEDULED
|
Upgrades of Arnes backbone. |
|
|
| NGI_IT |
INFN-NAPOLI-ATLAS |
TIER2 |
2021-02-16 11:00 |
2021-02-17 19:00 |
OUTAGE
|
SCHEDULED
|
Shutdown for maintenance and upgrade of the electrical infrastructure |
|
|
| NGI_IT |
HEPHY-UIBK |
TIER2 |
2021-02-12 20:00 |
2021-02-22 09:30 |
OUTAGE
|
UNSCHEDULED
|
hardware replacement |
|
|
| NGI_UK |
UKI-SCOTGRID-ECDF |
TIER2 |
2021-02-12 17:00 |
2021-02-15 10:00 |
OUTAGE
|
UNSCHEDULED
|
Extension of downtime due to switch failure at datacenter |
|
|
| NGI_IT |
INFN-NAPOLI-ATLAS |
TIER2 |
2021-02-13 13:00 |
2021-02-15 11:25 |
OUTAGE
|
UNSCHEDULED
|
cooling failure |
|
|
| AsiaPacific |
INDIACMS-TIFR |
TIER2 |
2021-02-15 00:00 |
2021-02-15 14:39 |
OUTAGE
|
UNSCHEDULED
|
DPM upgrade to latest version Extended Part3 |
|
|
| NGI_DE |
UNI-FREIBURG |
TIER2 |
2021-02-15 08:30 |
2021-02-15 14:05 |
OUTAGE
|
UNSCHEDULED
|
air condition failure |
|
|
| NGI_FRANCE |
IN2P3-IRES |
TIER2 |
2021-02-18 09:00 |
2021-02-18 17:00 |
OUTAGE
|
SCHEDULED
|
System updates |
|
|
| NGI_DE |
wuppertalprod |
TIER2 |
2021-02-16 07:00 |
2021-02-19 09:59 |
OUTAGE
|
UNSCHEDULED
|
broken central cooling pipe made whole cooling system to fail - recovery difficult |
|
|
| NGI_UK |
UKI-LT2-QMUL |
TIER2 |
2021-02-17 10:00 |
2021-02-17 16:00 |
WARNING
|
UNSCHEDULED
|
firmware updates for Lustre storage nodes |
|
|
| NGI_UK |
UKI-NORTHGRID-SHEF-HEP |
TIER2 |
2021-02-23 17:00 |
2021-02-24 12:00 |
OUTAGE
|
SCHEDULED
|
Power work in the cluster building |
|
|
| NGI_IBERGRID |
CIEMAT-LCG2 |
TIER2 |
2021-02-19 15:00 |
2021-02-19 22:55 |
OUTAGE
|
SCHEDULED
|
Substitution of our main firewall system (hardware) |
|
|
| NGI_IT |
INFN-TORINO |
TIER2 |
2021-02-15 16:00 |
2021-02-19 00:00 |
OUTAGE
|
UNSCHEDULED
|
Cooling failure |
|
|
| NGI_IT |
INFN-NAPOLI-ATLAS |
TIER2 |
2021-02-17 19:00 |
2021-02-18 16:52 |
OUTAGE
|
UNSCHEDULED
|
Shutdown for maintenance and upgrade of the electrical infrastructure (continued) |
|
|
| AsiaPacific |
TW-FTT |
TIER2 |
2021-02-21 06:00 |
2021-02-21 12:00 |
OUTAGE
|
SCHEDULED
|
unexpected dual submarine cable maintenance |
|
|
| Russia |
ITEP |
TIER2 |
2021-02-20 18:00 |
2021-02-21 19:00 |
OUTAGE
|
UNSCHEDULED
|
The system disk repair. |
|
|
| Russia |
ITEP |
TIER2 |
2021-02-21 18:50 |
2021-02-24 18:50 |
OUTAGE
|
SCHEDULED
|
The system disk repair. |
|
|
| NGI_IT |
Hephy-Vienna |
TIER2 |
2021-02-24 12:00 |
2021-02-24 13:00 |
OUTAGE
|
SCHEDULED
|
storage systems will be offline due to hardware maintenance |
|
|
| NGI_DE |
RWTH-Aachen |
TIER2 |
2021-02-24 09:00 |
2021-02-24 10:00 |
WARNING
|
SCHEDULED
|
Puppet migration |
|
|
| NGI_UK |
UKI-LT2-QMUL |
TIER2 |
2021-02-23 06:50 |
2021-02-23 07:20 |
WARNING
|
UNSCHEDULED
|
firmware updates for university network infrastructure |
|
|
| NGI_IT |
INFN-NAPOLI-ATLAS |
TIER2 |
2021-02-23 01:30 |
2021-02-23 17:35 |
OUTAGE
|
UNSCHEDULED
|
Cooling outage |
|
|
| NGI_IBERGRID |
ifae |
TIER2 |
2021-02-23 12:00 |
2021-02-23 13:00 |
WARNING
|
UNSCHEDULED
|
Network configuration change |
|
|
| NGI_UK |
UKI-SOUTHGRID-BRIS-HEP |
TIER2 |
2021-03-04 08:00 |
2021-03-05 08:00 |
OUTAGE
|
SCHEDULED
|
Site Network Maintenance |
|
|
| NGI_UK |
UKI-SOUTHGRID-BRIS-HEP |
TIER2 |
2021-03-05 08:00 |
2021-03-10 16:00 |
OUTAGE
|
SCHEDULED
|
(Possible fallout from) Site Network Maintenance |
|
|
| NGI_FRANCE |
IN2P3-IRES |
TIER2 |
2021-03-03 06:00 |
2021-03-04 06:00 |
OUTAGE
|
SCHEDULED
|
Infrastructure update |
|
|
| NGI_FRANCE |
IN2P3-IRES |
TIER2 |
2021-03-04 06:00 |
2021-03-05 06:00 |
OUTAGE
|
SCHEDULED
|
Infrastructure update |
|
|
| NGI_FRANCE |
IN2P3-IRES |
TIER2 |
2021-03-05 06:00 |
2021-03-06 06:00 |
OUTAGE
|
SCHEDULED
|
Infrastructure update |
|
|
| AsiaPacific |
IN-DAE-VECC-02 |
TIER2 |
2021-02-22 06:31 |
2021-02-25 06:30 |
OUTAGE
|
UNSCHEDULED
|
Migration of Kolkata-Cream (PBS) to Kolkata (HTCondor). |
|
|
| Russia |
RRC-KI |
TIER2 |
2021-02-24 13:00 |
2021-02-25 13:45 |
OUTAGE
|
UNSCHEDULED
|
Hardware failure |
|
|
| Russia |
RRC-KI |
TIER2 |
2021-02-25 13:45 |
2021-03-03 23:59 |
OUTAGE
|
SCHEDULED
|
Hardware failure |
|
|
| ROC_Canada |
CA-VICTORIA-WESTGRID-T2 |
TIER2 |
2021-02-25 17:00 |
2021-02-26 02:00 |
OUTAGE
|
UNSCHEDULED
|
rebuild k8s cluster |
|
|
| AsiaPacific |
IN-DAE-VECC-02 |
TIER2 |
2021-02-25 06:30 |
2021-03-01 12:00 |
OUTAGE
|
UNSCHEDULED
|
Migration of Kolkata-Cream (PBS) to Kolkata (HTCondor). |
|
|
| Russia |
ITEP |
TIER2 |
2021-02-25 16:35 |
2021-02-26 20:00 |
OUTAGE
|
UNSCHEDULED
|
The problems with ARC after upgrade to 6.9.0-1 version. |
|
|
| NGI_PL |
CYFRONET-LCG2 |
TIER2 |
2021-02-28 23:00 |
2021-03-30 22:59 |
WARNING
|
SCHEDULED
|
Decommissioning |
|
|