All systems operational

Show current status

Resolved
Instability on Optimism Goerli

Started
January 25, 2023 at 8:41 PM
Status
Resolved after about 3 hours

Impact

Operational
Affected
Goerli
Transaction Sequencing
  • Resolved
    Resolved

    We have successfully re-synced the sequencer, and re-enabled transaction ingress. We are monitoring for further issues.

    Re-syncing the sequencer caused a large chain reorganization on L2. If you are running an archival Optimism Goerli node, no action is necessary - the reorg was processed automatically. If you are running a full node, you will need to re-sync your node from the migrated Bedrock Goerli data directory.

    Infrastructure providers may need some time to re-sync their nodes as a result of this issue. See their respective status pages for more information.

    We’re sorry for any frustrations this outage caused our community, and are committed to sharing a post-mortem soon.

  • Identified
    Identified

    Our team has discovered that the earlier provider outage triggered a bug in an Optimism node, which caused syncing from Optimism and L1 to stop. Our team is currently syncing the sequencer, which is a process we anticipate could take up to several hours. Transactions submitted during the last two hours may be replayed if they are still in our transaction pool, otherwise they will be dropped.

    We will be working with our infra partners to ensure they’re able to get Optimism nodes up and running as quickly as possible, and will update again here when we have new information or full resolution. In the meantime, there is no action required from users.

  • Monitoring
    Update

    Transaction ingress on Goerli has been disabled while the sequencer catches up. We expect service to be restored within 15 minutes.

  • Monitoring
    Update

    We're still seeing degraded performance on Goerli. We are working on a fix.

  • Monitoring
    Monitoring

    We have deployed a fix for the Goerli testnet, and are monitoring the result. Nodes may need a few minutes to come back in sync.

    Mainnet has recovered.

  • Investigating
    Investigating

    One of our upstream RPC providers encountered an error. We're switching our infrastructure over to a new provider.

  • Identified
    Identified

    Service to the mainnet public endpoint has been restored.