Ticket #86 (closed defect: fixed)

Opened 10 years ago

Last modified 10 years ago

Claim fails for dome-broker

Reported by: irwin@… Owned by: varun
Priority: blocker Milestone:
Component: Clearinghouse Version:
Keywords: umass, vise, dome Cc:

Description

I am getting a certificate invalid error on my end when I try to execute a claim from dome-broker on geni.renci.org. The full stack trace I see from geni.cs.umass.edu is below. Below that is the stack trace from geni.renci.org that got printed on my screen. I am exporting from dome-site at geni.cs.umass.edu to dome-broker at geni.renci.org. I think the certificates are mis-matched The web addresses must be right, since the claim calls make it over to my end.

I sent the certificates via email to Varun and will not post them in this message.

Also, we're not running the most up to date version of the code.
Could that be a problem? What version should we be running?

From geni.cs.umass.edu:

2009-10-09 11:19:32,499 [Thread-73] INFO
orca.tools.authmodule.OrcaLoggingHandler? - <?xml version='1.0'
encoding='utf-8'?><soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Header
/><soapenv:Body><soapenv:Fault><faultcode>soapenv:Client</faultcode><faultstring>WSDoAllReceiver:
security processing
failed</faultstring><detail><Exception>org.apache.axis2.AxisFault?:
WSDoAllReceiver: security processing failed; nested exception is:

org.apache.ws.security.WSSecurityException: The signature

verification failed (The provided certificate is invalid)

at org.apache.rampart.handler.WSDoAllReceiver.processBasic(WSDoAllReceiver.java:259)
at org.apache.rampart.handler.WSDoAllReceiver.processMessage(WSDoAllReceiver.java:91)
at org.apache.rampart.handler.WSDoAllHandler.invoke(WSDoAllHandler.java:74)
at org.apache.axis2.engine.Phase.invoke(Phase.java:382)
at org.apache.axis2.engine.AxisEngine?.invoke(AxisEngine?.java:522)
at org.apache.axis2.engine.AxisEngine?.receive(AxisEngine?.java:487)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:328)
at org.apache.axis2.transport.http.AxisServlet?.doPost(AxisServlet?.java:254)
at javax.servlet.http.HttpServlet?.service(HttpServlet?.java:709)
at javax.servlet.http.HttpServlet?.service(HttpServlet?.java:802)
at org.apache.catalina.core.ApplicationFilterChain?.internalDoFilter(ApplicationFilterChain?.java:252)
at org.apache.catalina.core.ApplicationFilterChain?.doFilter(ApplicationFilterChain?.java:173)
at org.apache.catalina.core.StandardWrapperValve?.invoke(StandardWrapperValve?.java:213)
at org.apache.catalina.core.StandardContextValve?.invoke(StandardContextValve?.java:178)
at org.apache.catalina.authenticator.AuthenticatorBase?.invoke(AuthenticatorBase?.java:432)
at org.apache.catalina.core.StandardHostValve?.invoke(StandardHostValve?.java:126)
at org.apache.catalina.valves.ErrorReportValve?.invoke(ErrorReportValve?.java:105)
at org.apache.catalina.core.StandardEngineValve?.invoke(StandardEngineValve?.java:107)
at org.apache.catalina.connector.CoyoteAdapter?.service(CoyoteAdapter?.java:148)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at org.apache.tomcat.util.net.PoolTcpEndpoint?.processSocket(PoolTcpEndpoint?.java:527)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread?.runIt(LeaderFollowerWorkerThread?.java:80)
at org.apache.tomcat.util.threads.ThreadPool?$ControlRunnable?.run(ThreadPool?.java:684)
at java.lang.Thread.run(Thread.java:595)

Caused by: org.apache.ws.security.WSSecurityException: The signature
verification failed (The provided certificate is invalid)

at org.apache.ws.security.processor.SignatureProcessor?.verifyXMLSignature(SignatureProcessor?.java:257)
at org.apache.ws.security.processor.SignatureProcessor?.handleToken(SignatureProcessor?.java:79)
at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:279)
at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:201)
at org.apache.rampart.handler.WSDoAllReceiver.processBasic(WSDoAllReceiver.java:256)
... 24 more

</Exception></detail></soapenv:Fault></soapenv:Body></soapenv:Envelope>
2009-10-09 11:19:32,499 [Thread-73] INFO
orca.tools.authmodule.OrcaLoggingHandler? - <?xml version='1.0'
encoding='utf-8'?><soapenv:Envelope
xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"><soapenv:Header
/><soapenv:Body><soapenv:Fault><faultcode>soapenv:Client</faultcode><faultstring>WSDoAllReceiver:
security processing
failed</faultstring><detail><Exception>org.apache.axis2.AxisFault?:
WSDoAllReceiver: security processing failed; nested exception is:

org.apache.ws.security.WSSecurityException: The signature

verification failed (The provided certificate is invalid)

at org.apache.rampart.handler.WSDoAllReceiver.processBasic(WSDoAllReceiver.java:259)
at org.apache.rampart.handler.WSDoAllReceiver.processMessage(WSDoAllReceiver.java:91)
at org.apache.rampart.handler.WSDoAllHandler.invoke(WSDoAllHandler.java:74)
at org.apache.axis2.engine.Phase.invoke(Phase.java:382)
at org.apache.axis2.engine.AxisEngine?.invoke(AxisEngine?.java:522)
at org.apache.axis2.engine.AxisEngine?.receive(AxisEngine?.java:487)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:328)
at org.apache.axis2.transport.http.AxisServlet?.doPost(AxisServlet?.java:254)
at javax.servlet.http.HttpServlet?.service(HttpServlet?.java:709)
at javax.servlet.http.HttpServlet?.service(HttpServlet?.java:802)
at org.apache.catalina.core.ApplicationFilterChain?.internalDoFilter(ApplicationFilterChain?.java:252)
at org.apache.catalina.core.ApplicationFilterChain?.doFilter(ApplicationFilterChain?.java:173)
at org.apache.catalina.core.StandardWrapperValve?.invoke(StandardWrapperValve?.java:213)
at org.apache.catalina.core.StandardContextValve?.invoke(StandardContextValve?.java:178)
at org.apache.catalina.authenticator.AuthenticatorBase?.invoke(AuthenticatorBase?.java:432)
at org.apache.catalina.core.StandardHostValve?.invoke(StandardHostValve?.java:126)
at org.apache.catalina.valves.ErrorReportValve?.invoke(ErrorReportValve?.java:105)
at org.apache.catalina.core.StandardEngineValve?.invoke(StandardEngineValve?.java:107)
at org.apache.catalina.connector.CoyoteAdapter?.service(CoyoteAdapter?.java:148)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at org.apache.tomcat.util.net.PoolTcpEndpoint?.processSocket(PoolTcpEndpoint?.java:527)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread?.runIt(LeaderFollowerWorkerThread?.java:80)
at org.apache.tomcat.util.threads.ThreadPool?$ControlRunnable?.run(ThreadPool?.java:684)
at java.lang.Thread.run(Thread.java:595)

Caused by: org.apache.ws.security.WSSecurityException: The signature
verification failed (The provided certificate is invalid)

at org.apache.ws.security.processor.SignatureProcessor?.verifyXMLSignature(SignatureProcessor?.java:257)
at org.apache.ws.security.processor.SignatureProcessor?.handleToken(SignatureProcessor?.java:79)
at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:279)
at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:201)
at org.apache.rampart.handler.WSDoAllReceiver.processBasic(WSDoAllReceiver.java:256)
... 24 more

</Exception></detail></soapenv:Fault></soapenv:Body></soapenv:Envelope>
2009-10-09 11:19:32,500 [Thread-73] ERROR orca.dome-site - An error in
updateTicket(WSDoAllReceiver: security processing failed)
[####org.apache.axis2.AxisFault?: WSDoAllReceiver: security processing
failed

at org.apache.axis2.description.OutInAxisOperationClient?.send(OutInAxisOperation?.java:271)
at org.apache.axis2.description.OutInAxisOperationClient?.execute(OutInAxisOperation?.java:202)
at orca.shirako.proxies.soapaxis2.services.ActorServiceStub?.updateTicket(ActorServiceStub?.java:441)
at orca.shirako.proxies.soapaxis2.SoapAxis?2Return.updateTicket(SoapAxis?2Return.java:84)
at orca.shirako.kernel.BrokerReservation?.generateUpdate(BrokerReservation?.java:269)
at orca.shirako.kernel.BrokerReservation?.serviceProbe(BrokerReservation?.java:773)
at orca.shirako.kernel.Kernel.probePending(Kernel.java:753)
at orca.shirako.kernel.Kernel.tick(Kernel.java:1215)
at orca.shirako.kernel.KernelWrapper?.tick(KernelWrapper?.java:737)
at orca.shirako.core.Actor.externalTick(Actor.java:357)
at orca.shirako.kernel.RealtimeTick?$TickWrapper?.run(RealtimeTick?.java:137)

Caused by: java.lang.Exception: org.apache.axis2.AxisFault?:
WSDoAllReceiver: security processing failed; nested exception is:

org.apache.ws.security.WSSecurityException: The signature

verification failed (The provided certificate is invalid)

at org.apache.rampart.handler.WSDoAllReceiver.processBasic(WSDoAllReceiver.java:259)
at org.apache.rampart.handler.WSDoAllReceiver.processMessage(WSDoAllReceiver.java:91)
at org.apache.rampart.handler.WSDoAllHandler.invoke(WSDoAllHandler.java:74)
at org.apache.axis2.engine.Phase.invoke(Phase.java:382)
at org.apache.axis2.engine.AxisEngine?.invoke(AxisEngine?.java:522)
at org.apache.axis2.engine.AxisEngine?.receive(AxisEngine?.java:487)
at org.apache.axis2.transport.http.HTTPTransportUtils.processHTTPPostRequest(HTTPTransportUtils.java:328)
at org.apache.axis2.transport.http.AxisServlet?.doPost(AxisServlet?.java:254)
at javax.servlet.http.HttpServlet?.service(HttpServlet?.java:709)
at javax.servlet.http.HttpServlet?.service(HttpServlet?.java:802)
at org.apache.catalina.core.ApplicationFilterChain?.internalDoFilter(ApplicationFilterChain?.java:252)
at org.apache.catalina.core.ApplicationFilterChain?.doFilter(ApplicationFilterChain?.java:173)
at org.apache.catalina.core.StandardWrapperValve?.invoke(StandardWrapperValve?.java:213)
at org.apache.catalina.core.StandardContextValve?.invoke(StandardContextValve?.java:178)
at org.apache.catalina.authenticator.AuthenticatorBase?.invoke(AuthenticatorBase?.java:432)
at org.apache.catalina.core.StandardHostValve?.invoke(StandardHostValve?.java:126)
at org.apache.catalina.valves.ErrorReportValve?.invoke(ErrorReportValve?.java:105)
at org.apache.catalina.core.StandardEngineValve?.invoke(StandardEngineValve?.java:107)
at org.apache.catalina.connector.CoyoteAdapter?.service(CoyoteAdapter?.java:148)
at org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
at org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
at org.apache.tomcat.util.net.PoolTcpEndpoint?.processSocket(PoolTcpEndpoint?.java:527)
at org.apache.tomcat.util.net.LeaderFollowerWorkerThread?.runIt(LeaderFollowerWorkerThread?.java:80)
at org.apache.tomcat.util.threads.ThreadPool?$ControlRunnable?.run(ThreadPool?.java:684)
at java.lang.Thread.run(Thread.java:595)

Caused by: org.apache.ws.security.WSSecurityException: The signature
verification failed (The provided certificate is invalid)

at org.apache.ws.security.processor.SignatureProcessor?.verifyXMLSignature(SignatureProcessor?.java:257)
at org.apache.ws.security.processor.SignatureProcessor?.handleToken(SignatureProcessor?.java:79)
at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:279)
at org.apache.ws.security.WSSecurityEngine.processSecurityHeader(WSSecurityEngine.java:201)
at org.apache.rampart.handler.WSDoAllReceiver.processBasic(WSDoAllReceiver.java:256)
... 24 more

at org.apache.axis2.AxisFault?.<init>(AxisFault?.java:159)
... 11 more

####]
2009-10-09 11:19:32,500 [Thread-73] ERROR orca.dome-site - remote
error for reservation: res: #1fc55bb2-bf44-43d2-9c92-d07f808f8431
slice: dome-slice Ticketed None rset: units 1 concrete 1 Ticket
[units = 1 oldUnits = 1 Slice=dome-slice] term=[58:58:60000058]:
Callback failed: org.apache.axis2.AxisFault?: WSDoAllReceiver: security
processing failed

From geni.renci.org:

An error occurred during reserve for reservation #6FEC602C Exception
stack trace: orca.shirako.kernel.Kernel.error(Kernel.java:363)
orca.shirako.kernel.Kernel.reserve(Kernel.java:1133)
orca.shirako.kernel.KernelWrapper?.handleReserve(KernelWrapper?.java:502)
orca.shirako.kernel.KernelWrapper?.ticket(KernelWrapper?.java:768)
orca.shirako.core.Broker.claim(Broker.java:227)
orca.manage.extensions.standard.actors.AgentManagerObject?.claimResources(AgentManagerObject?.java:349)
orca.manage.extensions.standard.actors.proxies.local.LocalAgentManagementProxy?.claimResources(LocalAgentManagementProxy?.java:76)
sun.reflect.NativeMethodAccessorImpl?.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl?.invoke(NativeMethodAccessorImpl?.java:39)
sun.reflect.DelegatingMethodAccessorImpl?.invoke(DelegatingMethodAccessorImpl?.java:25)
java.lang.reflect.Method.invoke(Method.java:585)
org.apache.velocity.util.introspection.UberspectImpl?$VelMethodImpl?.invoke(UberspectImpl?.java:267)
org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:197)
org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:175)
org.apache.velocity.runtime.parser.node.ASTReference.value(ASTReference.java:327)
org.apache.velocity.runtime.parser.node.ASTExpression.value(ASTExpression.java:51)
org.apache.velocity.runtime.parser.node.ASTSetDirective.render(ASTSetDirective.java:95)
org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:55)
org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:70)
org.apache.velocity.runtime.parser.node.SimpleNode?.render(SimpleNode?.java:230)
org.apache.velocity.Template.merge(Template.java:256)
org.apache.velocity.tools.view.servlet.VelocityLayoutServlet?.mergeTemplate(Unknown
Source) org.apache.velocity.tools.view.servlet.VelocityViewServlet?.doRequest(Unknown
Source) org.apache.velocity.tools.view.servlet.VelocityViewServlet?.doPost(Unknown
Source) javax.servlet.http.HttpServlet?.service(HttpServlet?.java:709)
javax.servlet.http.HttpServlet?.service(HttpServlet?.java:802)
org.apache.catalina.core.ApplicationFilterChain?.internalDoFilter(ApplicationFilterChain?.java:252)
org.apache.catalina.core.ApplicationFilterChain?.doFilter(ApplicationFilterChain?.java:173)
org.apache.catalina.core.StandardWrapperValve?.invoke(StandardWrapperValve?.java:213)
org.apache.catalina.core.StandardContextValve?.invoke(StandardContextValve?.java:178)
org.apache.catalina.authenticator.AuthenticatorBase?.invoke(AuthenticatorBase?.java:524)
org.apache.catalina.core.StandardHostValve?.invoke(StandardHostValve?.java:126)
org.apache.catalina.valves.ErrorReportValve?.invoke(ErrorReportValve?.java:105)
org.apache.catalina.core.StandardEngineValve?.invoke(StandardEngineValve?.java:107)
org.apache.catalina.connector.CoyoteAdapter?.service(CoyoteAdapter?.java:148)
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:869)
org.apache.coyote.http11.Http11BaseProtocol$Http11ConnectionHandler.processConnection(Http11BaseProtocol.java:664)
org.apache.tomcat.util.net.PoolTcpEndpoint?.processSocket(PoolTcpEndpoint?.java:527)
org.apache.tomcat.util.net.LeaderFollowerWorkerThread?.runIt(LeaderFollowerWorkerThread?.java:80)
org.apache.tomcat.util.threads.ThreadPool?$ControlRunnable?.run(ThreadPool?.java:684)
java.lang.Thread.run(Thread.java:595)

Change History

Changed 10 years ago by varun

  • keywords umass, vise, dome added
  • status changed from new to accepted

Things we checked:

  1. checked all certificates/guids for inconsistencies
  2. checked version of keytool - we use the sun version because the gnu version is weirdly incompatible.
  3. checked dates/times on the certificates.
  4. checked for differences in the runtime environment in the jails

So far everything looks okay. Still waiting to see what the differences in the code between the code running at Umass and the head of Anacortes are.

Changed 10 years ago by ibaldin

  • status changed from accepted to closed
  • resolution set to fixed

The issue was the expired admin certificate, which was renewed. New sets of actor certificates have been installed also.

Note: See TracTickets for help on using tickets.