Operations Portal

This service is provided by CCIN2P3, and funded by the European Union’s Horizon 2020 research and innovation programme

Latest news

• GOCDB returned to read/write mode

January 18, 2023 12:50 | Adrian Coveney
More details

GOCDB returned to read/write mode

Dear all,

The DNS issues at RAL have been resolved, so https://goc.egi.eu/ is now being served by the primary again and has returned to read/write mode.

It may take time for some connections to migrate back (i.e. still showing READ ONLY MODE in the portal).

Thanks,
The GOCDB Team

January 18, 2023 12:50
by Adrian Coveney

• se.igfae.usc.es (DOME access endpoint at USC-LCG2) will be decommissioned

November 24, 2022 15:20 | Marcos Seco Miguelez
More details

se.igfae.usc.es (DOME access endpoint at USC-LCG2) will be decommissioned

Our DOME access endpoint, se.igfae.usc.es, will be decommissioned in the following weeks.

Given the amount of space offered by our SE it is unlikely that LHCb VO will be affected by this operation. Should this not be the case, please let us know before January 15th 2023 so we can extend the foreseen downtime interval. A GGUS ticket, https://ggus.eu/index.php?mode=ticket_info&ticket_id=159611, to follow the decommission procedure has been created and has more information about the timeline.

No new SE will be available at our site since the current one have seldom been used.

November 24, 2022 15:20
by Marcos Seco Miguelez

• DESY VOMS server certificate migration completed

November 21, 2022 14:48 | Thomas Hartmann
More details

DESY VOMS server certificate migration completed

Dear Users of VOs using the DESY VOMS server,

as previously announced, we have completed the update of the host certificate of the VOMS server at
grid.desy.de
where your VO is hosted. As the host certificate details have changed, an update to the lsc and vomeses files is necessary to obtain new proxies without errors. If you encounter errors while obtaining a new grid proxy, please contact your resource admin to update the locally installed voms details.

Updated server credentials can be found as rpm or tarball under
https://gitlab.desy.de/grid/voms-cert-transition
The rpm package `desy-voms-all` with the new host certificate details can also be found in the WLCG repositories.

Also updated credential files have been propagated through the /cvmfs/grid.cern.ch repository for users of CVMFS based environments.

For OSG users, an updated vo roles package is available in the osg-testing repository, which is expected to move to the osg repository during the day.

If you have any questions, please do not hesitate to contact us.

Cheers,
Thomas for the DESY Grid team

November 21, 2022 14:48
by Thomas Hartmann

• DESY VOMS server downtime/update on Monday, Nov.21, requiring client updates

November 15, 2022 11:14 | Thomas Hartmann
More details

DESY VOMS server downtime/update on Monday, Nov.21, requiring client updates

Dear Users of the VOMS instance hosted at DESY,

coming week on Monday, November 21, the host certificate of the VOMS server
grid-voms.desy.de
will be updated for changing to a new CA.

With the instance's new host certificate, clients will need to update their vomses and lsc files to trust the changed DN.

Without updated VOMS descriptions, clients will fail to obtain new proxies.

If necessary, please update your clients during the downtime accordingly.

Updated rpm, lsc and vomses files as well as a more detailed description are available under
https://gitlab.desy.de/grid/voms-cert-transition

For any questions or suggestions, please contact us under
grid@desy.de

See also the downtime announcement under
https://goc.egi.eu/portal/index.php?Page_Type=Downtime&id=33173

Regards,
Thomas for the DESY Grid team

November 15, 2022 11:14
by Thomas Hartmann

• VOMS server downtime/update on Monday, Nov.21, requiring client updates

November 14, 2022 17:28 | Thomas Hartmann
More details

VOMS server downtime/update on Monday, Nov.21, requiring client updates

Dear Users of the VOMS instances hosted at DESY,

coming week on Monday, November 21, the host certificate for the VOMS server
grid-voms.desy.de
will be updated for changing to a new CA.

With the instance's new host certificate, clients will need to update their vomses and lsc files to trust the changed DN.

Without updated VOMS descriptions, clients will fail to obtain new proxies.

If necessary, please update your clients during the downtime accordingly.

Updated rpm, lsc and vomses files as well as a more detailed description are available under
https://gitlab.desy.de/grid/voms-cert-transition

For any questions or suggestions, please contact us under
grid@desy.de

See also the downtime announcement under
https://goc.egi.eu/portal/index.php?Page_Type=Downtime&id=33173

Regards,
Thomas for the DESY Grid team

November 14, 2022 17:28
by Thomas Hartmann

More news

• Decommissioning of DPM / SRM at UKI-SCOTGRID-GLASGOW

October 19, 2022 14:14 | Gordon Stewart
More details

Decommissioning of DPM / SRM at UKI-SCOTGRID-GLASGOW

Dear all,

We would like to announce that UKI-SCOTGRID-GLASGOW has begun the decommissioning procedure for its DPM and SRM services. A tracking ticket can be found here:

https://ggus.eu/index.php?mode=ticket_info&ticket_id=159276

Associated services will be placed into downtime from Wednesday 16th November. The service will be removed on Wednesday 30th November.

VO managers should aim to ensure any critical data has been transferred elsewhere by Wednesday 16th November.

We have discussed this with LHCb directly, as the major remaining user of this service, but we recommend that other VOs check whether they have critical data remaining in this service.

Regards,

Gordon Stewart
ScotGrid Technical Co-ordinator

October 19, 2022 14:14
by Gordon Stewart

• Decommissioning of DPM / SRM at UKI-SCOTGRID-GLASGOW

October 19, 2022 11:27 | Gordon Stewart
More details

Decommissioning of DPM / SRM at UKI-SCOTGRID-GLASGOW

Dear all,

We would like to announce that UKI-SCOTGRID-GLASGOW has begun the decommissioning procedure for its DPM and SRM services. A tracking ticket can be found here:

https://ggus.eu/index.php?mode=ticket_info&ticket_id=159276

Our plan is to switch off this service from Wednesday 16th November. We have discussed this with LHCb directly, as the major remaining user of this service, but we recommend that other VOs check that all critical data has been transferred elsewhere.

Regards,

Gordon Stewart
ScotGrid Technical Co-ordinator

October 19, 2022 11:27
by Gordon Stewart

• [Update] RAL-LCG2 including GOCDB in unscheduled downtime

October 18, 2022 18:05 | Alastair Dewhurst
More details

[Update] RAL-LCG2 including GOCDB in unscheduled downtime

Dear all,

As of 16:00 UTC, the network failure that caused the loss of the databases services at RAL has been resolved, although we are currently running in a non-resilient manner. We will be running at risk over night.

The GOCDB is back in full production (i.e. both writes and reads).

The RAL Tier-1 Disk storage (Echo) and Tape storage (Antares) are back in production. The batch farm will be brought back online tomorrow morning. Further downtimes will be noted in the GOCDB (rather than more announcements).

Thank you for your patience

Alastair

RAL Tier-1 Manager

October 18, 2022 18:05
by Alastair Dewhurst

• [Update] RAL-LCG2 including GOCDB in unscheduled downtime

October 18, 2022 13:06 | Alastair Dewhurst
More details

[Update] RAL-LCG2 including GOCDB in unscheduled downtime

Dear all,

The site downtime at RAL caused by the core network failure is still ongoing. We are working on recovering the databases currently.

As mentioned in the previous update, the GOCDB is working (at risk) in read-only mode: https://goc.egi.eu/

Echo (Disk storage) is functioning but in a degraded state.

Antares (Tape storage) is currently down however we can bring the service back quickly once the network has stabilized.

We are expecting the batch farm to remain down until tomorrow morning.

We will provide an update by 14:00 UTC and hope to be in a position to announce the return of some production services before the end of the day.

Thanks,

Alastair

RAL Tier-1 Manager

October 18, 2022 13:06
by Alastair Dewhurst

• Ongoing GOCDB unavailability

October 18, 2022 11:02 | Greg Corbett
More details

Ongoing GOCDB unavailability

Dear all,

The network issues at RAL are still ongoing. Overnight, partial network connectivity was restored but this is not sufficient to restore the primary (read/write) instance of GOCDB.

https://goc.egi.eu/ is currently being served by the read-only failover, however this remains at risk while network recovery work is ongoing to restore services at RAL.

The failover remains directly accessible: https://gocdb.hartree.stfc.ac.uk.

A further update will be given by 12:00 UTC.

Thanks,
The GOCDB Team

October 18, 2022 11:02
by Greg Corbett

• RAL-LCG2 including GOCDB in unscheduled downtime

October 17, 2022 20:07 | Alastair Dewhurst
More details

RAL-LCG2 including GOCDB in unscheduled downtime

Dear All

At shortly after 15:00 local time (14:00 UTC) today (17/10/22) there was a major failure of the RAL core network. All services are currently down including the GOCDB which would normally be used to declare outages.

After 6 hours of work we have only been able to partially restore network connectivity and we are thus declaring a whole site outage until at least midday (11:00 UTC) tomorrow (18/10/22).

Alastair

RAL Tier-1 Manager

October 17, 2022 20:07
by Alastair Dewhurst

• Ongoing GOCDB unavailability

October 17, 2022 18:07 | Adrian Coveney
More details

Ongoing GOCDB unavailability

Due to network issues at RAL, GOCDB remains unavailable. Staff are on site investigating.

We will provide a further update once more information is available, but that may not be until tomorrow. Connectivity to GOCDB may be restored in the meantime.

October 17, 2022 18:07
by Adrian Coveney

• GOCDB 5.10.0 Update

October 12, 2022 12:54 | Greg Corbett
More details

GOCDB 5.10.0 Update

Dear all,

The GOCDB team will soon be releasing version 5.10.0 into production, which brings necessary changes relating to improving user privacy. This new version is already running on our preproduction system [1] (data updated 3rd Oct 2022) and there is a GGUS ticket [2] to track any issues with it.

Once version 5.10.0 is deployed into production, which is scheduled for week commencing 31st Oct 2022, users who have not visited the portal [3] in 18 months will have their GOCDB accounts automatically deleted. Going forward, warning emails will be sent monthly to users with 12 months or more of inactivity.

If you are concerned about your account being deleted, we recommend you visit the GOCDB portal [3] whilst authenticated to ensure your account is considered active.

Thanks,
The GOCDB team.

[1] https://gocdb-preprod.egi.eu/
[2] https://ggus.eu/index.php?mode=ticket_info&ticket_id=159180
[3] https://goc.egi.eu/portal/

October 12, 2022 12:54
by Greg Corbett

• EGI monthly broadcast - October 2022

October 3, 2022 12:36 | Alessandro Paolini
More details

EGI monthly broadcast - October 2022

=============== Content: ==================

1) Release of UMD 4.17.0 and 4.17.1

2) New configuration for monitoring webdav endpoints

3) New configuration for monitoring XrootD (and EOS) endpoints

============================================

1) Release of UMD 4.17.0 and 4.17.1

UMD 4.17.0 released: https://repository.egi.eu/UMD/4.17.0.html
- ArC 6.1.5.1
- Myproxy 6.2.14
- gridFTP 13.26.1
- xroot 5.4.2
- condor 9.0.16,
- htcondor-CE 5.1.3
- dCache 6.2.44
- gfal2 clients 2.20.5
- davix 0.8.2,
- dmlite 1.15.2
- cvmfs 2.9.4
- egi-cvmfs 4.4.0
- new nagios probes for ondedata and IM

UMD 4.17.1 released: https://repository.egi.eu/UMD/4.17.1.html
- dCache 7.2.15: new major release
- Argus 1.7.5: bug fix release that prevents argus pepd crash if non standard characters are found in the DN Certification Authorities released by the EGI Trust Anchor team.
- glite-infoprovider-ldap 1.5.0: bug fix release that suppress the software and job information to be added to the topbdii01 ldap preventing huge memory consuption.

2) New configuration for monitoring webdav endpoints

It was agreed a change on the GOCDB fields to use for monitoring the webdav endpoints.
For some time the webdav metric, which is in the critical profile, will take into account either the old or the new settings in order to guarantee a smooth transition to the new settings.

Please follow the indications as reported in https://docs.egi.eu/internal/configuration-database/adding-service-endpoint/#webdav

The endpoint url information used for monitoring purposes should be set in the extension properties section. Create the following:
- Name: ARGO_WEBDAV_OPS_URL
- Value: webdav URL containing also the VO ops folder

3) New configuration for monitoring XrootD (and EOS) endpoints

We would like to remind you to enable the ops VO on your EOS and Xrootd endpoints in order to allow the execution of the monitoring probe (see https://github.com/EGI-Federation/nagios-plugins-xrootd ) introduced some months ago.

On GOCDB the EOS endpoints are registered as XrootD service type hence these indications apply for any endpoint associated to XrootD service type.

It was agreed a change on the GOCDB fields to use for monitoring the XrootD endpoints, so please follow the guide in https://docs.egi.eu/internal/configuration-database/adding-service-endpoint/#eos-and-xrootd-service-endpoints

The endpoint url information used for monitoring purposes should be set in the extension properties section. Create the following:
- Name: ARGO_XROOTD_OPS_URL
- Value: XRootD base SURL to test (the path where ops VO has write access)

October 3, 2022 12:36
by Alessandro Paolini

Home

Broadcast

Downtimes

VO Management

VO List

Dashboard

Metrics

Sla

v6.5 - 2022-10-05