RG063 – Webservices for fun and profit, part 1

By Max Ranzau

Part 2 of this article is here.

soapboxFrom the SOAP box Dept. As of recent I’ve been engaged in a couple of interesting projects, involving SOAP web interfaces and IT Store services. One of my clients is looking to configure their firewalls, while others are aiming at configuring their infrastructure. Another pet project is interfacing with payment processors to be able to handle credit card transactions, all this while providing a simple interface towards the organization. The advantages are obvious: The users are kept miles away from any administration interface, while IT doesn’t have to lift a finger as everything is automatic.

In the past we needed usually needed to fire off SSH commands to talk to routers and such, however modern cloud/infrastructure components will usually offer a web-based API to configure everything.

asmithEssentially this article describes how to leverage the ability of RES technology to convert an XML based web interface into a service that can be presented to regular users. This opens up the opportunity for self-service access-management with approval workflows, for example someone needs to have FTP access for the rest of the day. For obvious reasons, we don’t want the requester (probably being a regular user, or worse – a developer ;) within a mile of the management interface, on the other hand we don’t want to have someone in IT standing by waiting to do a manual configuration process, that could be handled automatically. To paraphrase Agent Smith in the Matrix: “Never send a human to do a machine’s job, Mr. Andersssson..

Many vendors have their Web-API up and running, yet may not offer a clear path to utilize it’s full potential, which leaves a gap between the actual capability and the ability to reach it. This article will show you how to open up a public accessible Web-API and work with it from Automation Manager. In part 2, we are going to look at some nifty methods to build a RES IT Store service up around it – Let’s get started:

Shanghai-MenuThe WSDL file. This is the first thing you’ll need in order to get anywhere. If you are wondering what a WSDL file is, the full wiki is here, but for conceptual use, just think of it as the menu card for a restaurant. A WSDL describes what’s available, the so-called methods. The methods can either return information (a query) or do something for you (a task), very similar to what we know from Automation Manager. Speaking of the devil, you might be aware that AM actually has it’s own web-interface which you can hit on any dispatcher if enabled. We are however not going to touch on that in this article as it’s about controlling other web-services from AM.

Here’s the rub however: A WSDL file may or may not be available to you for the given SOAP service provider you want to work with. While some companies such as SalesForce.com provide a Web-API as part of their subscription, others may chose not to make it available, at least not without some sort of NDA. However, there are public SOAP example services available for testing purposes, google around and you’ll find several examples. For the purpose of today’s article, I’ve chosen to build an example around pulling a weather report. You’ll find it’s definition here and it’s WSDL URL here.

soap1The Automation. As of Automation Manager 2014, FR1 a built-in task, called WebService is available. This is where you specify the necessary WSDL file. The file can either be specified as a file (although not a AM resource) or residing somewhere on the internet, as a URL. I would suggest as a best practice to refer to a URL when possible, as new methods may be published by the SOAP vendor later down the road.

When you paste in the WSDL URL, Automation Manager will enumerate all the methods in the WDSL. Depending on the service, there can be several hundred methods which can take several minutes to enumerate. In the case of the weather service, there’s basically two calls available, one to return a list of cities for a specified country, and one to pull the curent weather from a specified city and country. You’ll note that there are two types of calls (soap) and (soap12) It’s just what version of SOAP you’re using and for this example it doesn’t matter. You can read more about the differences here.

As you can see, you’ll need to add an AM connector for no reason what so ever, as since of FR2 connectors aren’t licensed anymore. If you’re in doubt, you can go and check your setup|licensing node in AM and you’ll see that you are charged zero licenses for this.

Once you pick a method from the WSDL, you’ll need to fill in the method’s parameters, in order to tell it what to do. This is where the Automation Manager’s own parameter’s come in handy. However, if you are creating them manually, be sure that you use the proper AM parameter types, anyway we’ll get to that in a moment.

Testing: Automation Manager has a nifty Test-button, to ensure your web-service call is working as it should. As of AM I found a weird little snafu, as it seems the Test button doesn’t work, even if the default values for the module parameters are filled out. Anyway I’ve reported it and a quick workaround until RES gets around to fixing it is this: To check if the webservice works as advertized, just paste in the actual values directly into the fields, then hit the test button and finally  re-insert the AM parameters before you save. Update: I’ve been informed Nov 5th by RES Support that this will be fixed in AM 2014 SR3.

Note that you can’t just enter any old city. That’s why there’s a method provided to return available cities in a specified country. If you are wondering why Jerkwater, USA is not listed, and returns ‘No Data’ if specified, only cities with airports that has weather reporting, are available for query.

soapfailAw, damn! Since I built this example and begun writing this article, it’s become apparent that said weather-webservice isn’t exactly super-stable. While it often replies quick without hassle, at the time of writing, it lags 15-20 seconds before coming back with a timeout message This is not Automation Manager’s or your/ISP’s fault. It’s simply the given weather service that isn’t able to keep up with demand. So if you decide to check this out, you may get the result shown on the right. If that continues to be the case, either muster some patience and hit the Test button again or select a similar, different service.

soapsuccessAnyway if things go the way we all want them to, when you fill in a valid city and country for testing this webservice, ideally you should get return data looking something like this example on the right.

As you can see, either way – fail or success, the return data is wrapped in in a yummy roll of XML, which it’s our next job to decipher and somehow get lobbed over into Automation Manager parameters so we can work with them.

XML to AM Parameter Translation: This is where things get a little funky. While Automation Manager’s repository of string handling functions is nice, it is however not capable of dealing natively with XML to any meaningful degree. There is however something very capable of that, namely PowerShell, which Automation Manager natively can leverage. So without turning this example into a PowerShell scripting circus, we are only going to use PowerShell for what it can help us accomplish in this instance, which is getting the individual XML fields out.

We do this by using the AM PowerShell tasklet. It allows us to insert a PowerShell script in the task, and extracting the stdout to a parameter. The input to each PowerShell instance is the multi-line XML output from the WebService call. Since we have multiple parameters to fill, this unfortunately means that we will have to run PowerShell multiple times, once for every parameter. If somebody figures out a smarter way to do this, I’m all ears. Anyway, the secret sauce is having Powershell do the follwing for every parameter:

  1. PowerShell variable = XML output from Webservice call stored in $[WebResult]
  2. Print the XML sub-component we want such as Temperature, to stdout

In PowerShell this will look something like this:

$xmldata = [xml] @"
Write-Host $xmldata.CurrentWeather.Temperature

ps-taskThis is the point where we utilize Automation Manager’s ability to store the stdout of a PowerShell script into a Module Parameter. As mentioned, it looks like we can only currently do this one at a time, which is why we need to invoke the above script once per XML value we want to extract.

Now that we have the capability to pull information out of a web service and into a Automation Manager parameter, really the rest is up to you what you want to use the information for. In the next part of this article, we are going to look at how to use RES IT Store to build a service to utilize the information.

bb-examples-wsThe building-block. As per usual, here is an example building block for you to import and take for a spin. It contains 3 modules. One module just returns a single value, the name of which you specify with $[value]. Another module returns all the cities for a given country. This is necessary as there may not be a weather report available for the city you have in mind. As mentioned it’s usually for cities with an airport. The last module in this example pack illustrates how to use PowerShell to pull out all the weather data into separate AM parameters.

Click here to download the building block: legobrick-cropped

Proceed here to part 2, where we will build a IT Store service around it. If you like what you see or would like to know more about integrating your particular webservices with Automation Manager, feel free to reach out via the RCS contact page.

No Comments

No comments yet.

RSS feed for comments on this post. TrackBack URI

Leave a comment

Comments are welcome as always. Just do the math below. * Time limit is exhausted. Please reload the CAPTCHA.