Archive for August, 2012

My thoughts on SoapUI Pro as a Web Service Testing tool

I’ve been loosely working with SoapUI for over a year now and thought I would share some of my thoughts on what I think about it so far.  Overall, SoapUI Pro is a great tool that has made my testing team more productive and simplifies many of their common testing tasks.  Here are some items I would like to see on the roadmap or find out if/how others are working around these gaps.  Also, since i’m by no means an expert, i’m willing to stand corrected on some of my claims here.  I’d love to be proven wrong on these painpoints so i can improve my team’s productivity.

SoapUI Pro needs better support for version control systems (i.e subversion).

  • SoapUI Pro should allow you to do a check-in directly from the tool, and also have merge support so test cases, suites or projects can be merged in painlessly.  Currently, testers/developers have to leave the tool and make the check-in from a secondary tool.  This is inconvenient and not helpful to building a culture of continuous check-ins which support the goal of continuous integration.  It’s my understanding that there is an eclipse plug-in for SoapUI but not SoapUI Pro.  If this is the case, that does not make sense to me (since i’m a paying customer).  I would like to be shown otherwise but I have not been able to find info suggesting that an existing plugin has the Pro features.

SoapUI Pro has some usability issues that need to be flushed out.

  • For example, where is the undo feature?  If you make a mistake in SoapUI you are screwed (hope you checked in your work, see previous bullet point)
  • Making changes to properties does not necessarily propagate down the way you would suspect.  My best example of this is in the JDBC Connections tab you can edit the connection string which could be used by 1 or more test cases.  Once you edit the connection string directly you would think all the test steps would get updated but you would be wrong.  There is an additional step of ‘configuring’ the connection that needs to be performed that would make the test cases ‘aware’ of the change

SoapUI Pro needs better error messaging when running from the command line.

  • In our case we use have a Jenkins job which uses a python wrapper to execute the test runner from the command line.  If/when tests fail it is difficult to determine why they failed.  It’d be nice to see what service point it was pointing to, what database it was trying to connect to but instead it usually just lists a whole bunch of xpath errors.  I’m exploring working around this on our end by supplementing the test steps with better logging information, but similar to junit or other frameworks you should have the ability to plug in to a log4j library and get useful error information more easily

SoapUI Pro needs to flag disabled test steps as ignored or skipped similar to Junit style when reported on in Jenkins

  • This will allow the testers to have visibility to what is disabled and conduct test maintenance more easily.

SoapUI’s support for groovy is excellent

  • This is a very powerful way to expand the capabilities of the tool.  Well done!  Although  adding Python support would be even better 🙂

Overall, I’m a huge fan of the tool, it’s hard to find a tool better suited to testing web services that allows test cases to be maintained in an organized library, is easy to get up and running with minimal training and let’s you test web services end to end.

, ,

Leave a comment