Ticket #336 (closed task: fixed)

Opened 5 years ago

Last modified 5 years ago

"Unscheduled" Maintenance, 5/6/2014

Reported by: vjo Owned by: vjo
Priority: major Milestone:
Component: External: Testing and Redeployment Version: baseline
Keywords: Cc: yxin, jonmills, ibaldin

Description

We need to take an unscheduled maintenance, in order to resolve the issues with ExoSM.
At the same time, we will:

1) Upgrade ORCA code on ExoSM
2) Switch mysql on ExoSM to use a larger file store

Current open question for this maintenance:
Does ORCA need to be updated on any other portions of the ExoLayer? (i.e. BEN AM, NLR/ION AM)?

Change History

Changed 5 years ago by vjo

  • owner changed from vjo to yxin
  • component changed from Infrastructure: ExoGENI Racks ORCA to External: Testing and Redeployment

Changed 5 years ago by vjo

  • status changed from new to assigned
  • owner changed from yxin to vjo

Changed 5 years ago by yxin

yes, BEN and NLR control need to be updated to the latest.

Changed 5 years ago by vjo

  • cc ibaldin added

Changed 5 years ago by vjo

OK folks - 3 PM.
MySQL has been switched.
We've done some testing, and BEN is in a bad state.

What's the call on continued diagnosis, vs. cleaning up, restarting, and re-testing?

Just trying to figure out the endgame...

Changed 5 years ago by yxin

I'd suggest cleaning up and restart controller, NLR/ION, and BEN, to open it.

The BEN failure you had happened several successful mixed mp and p2p slices (from ORCA perspective), and I didn't have clue.

I also suggest RENCI people to avoid use RCI rack if possible after opening, to make it live longer...

-Yufeng

Changed 5 years ago by vjo

What's wrong w/ the RCI rack?

Changed 5 years ago by yxin

There is nothing wrong with the RCI rack. What I meant was to avoid using RCI rack in inter-domain slices to avoid using BEN, if possible, for your real experiments.

Changed 5 years ago by vjo

OK - slices on ExoSM on the way down.

Could somebody please make sure the BEN switches are clean for me?

Changed 5 years ago by vjo

ExoSM, BEN, and NLR/ION actors restarted.
Awaiting "all clear" on BEN switches to proceed with testing.

Changed 5 years ago by vjo

Switches all clear, testing complete.
Minor hiccup, due to BBN-w4 being brought online when it should not have been.

Testbed is back online; this maintenance is over.

Changed 5 years ago by vjo

  • status changed from assigned to closed
  • resolution set to fixed
Note: See TracTickets for help on using tickets.