IDM & the Holidays: Part V – Automated Transaction Monitoring/Reporting
As another year comes to an end and new holiday season is upon us, we reflect again on how an IDM solution can help to solve common holiday woes.
We’ve already looked at how to keep processes rolling even when staff are not in the office while they enjoy the holidays with their families (IDM & the Holidays).
We’ve discussed how an automated IDM solution can provide notifications to users about expiring passwords and grace logins (IDM & the Holidays: Part II).
Next, we looked at how an IDM solution can be of great value in processing the influx of new users that commonly are onboarded after the holiday break (IDM & the Holidays: Part III).
And last year, we discussed how an IDM solution can reduce the amount of staff required to support continued operations during the holidays where a large volume of staff may be on vacation to spend time with their families (IDM and the Holidays: Part IV).
This year, I’d like to talk about monitoring and issue reporting of your IDM environment.
Generally speaking, it is always a good idea to monitor the state and health of any IDM environment regularly to ensure consistent and proper processing. However, this becomes even more critical during the holidays when there is typically a smaller workforce across the enterprise to perform monitoring, issue investigations and issue resolutions.
Granted, there are multiple ways that an organization can choose to monitor their IDM environment through built-in services, 3rd party monitoring tools, and/or manual processes. And while many organizations will choose one or more processes to perform monitoring, this post will specifically focus on letting the NetIQ IDM drivers assist/perform some self-monitoring and notification processes that can help during holidays where the number or availability of system administrators/help desk personnel who generally track/resolve any IDM issues may be reduced.
Now, this is not to be confused with the NetIQ IDM drivers’ built-in health check capabilities. That is the ability to configure drivers to detect certain criteria such as cache size, number of transactions, etc. that result in a driver condition state (green, yellow, red).
This post is about adding custom code to drivers that will allow them to detect and take action when errors happen in individual transactions that might otherwise go unnoticed and prevent users from getting access to applications and systems as expected.
To do this, it is a two-part solution (focusing on capturing events in the Subscriber channel).
Part 1: Defining Returnable Data
The first part is partly optional and can vary from organization to organization, or even from driver to driver, based on each organization’s needs within each driver. This is where we define what information we would like for this driver to return, along with the transaction status (Error, Success, Retry, Warning, Fatal, etc.) so that additional details can be provided via the email notifications that come later.
On the Output Transform policy set, create a new policy (or you can use an existing policy if you like). In that policy, add the following rule (we will break it down after the example):
Which looks like this in XML:
<rule> <description>On Add Event add operation data</description> <conditions> <and> <if-class-name mode="nocase" op="equal">MSolUser</if-class-name> <if-operation mode="case" op="equal">add</if-operation> </and> </conditions> <actions> <do-if> <arg-conditions> <and> <if-xpath op="not-true">operation-data</if-xpath> </and> </arg-conditions> <arg-actions> <do-append-xml-element expression="." name="operation-data"/> </arg-actions> <arg-actions/> </do-if> <do-set-op-property name="parent-node-1"> <arg-string> <token-text xml:space="preserve">status</token-text> </arg-string> </do-set-op-property> <do-append-xml-element expression="operation-data" name="return-to-me"/> <do-append-xml-element expression="operation-data/return-to-me" name="command"/> <do-append-xml-text expression="operation-data/return-to-me/command"> <arg-string> <token-text xml:space="preserve">addUser</token-text> </arg-string> </do-append-xml-text> <do-append-xml-element expression="operation-data/return-to-me" name="srcdn"/> <do-append-xml-text expression="operation-data/return-to-me/srcdn"> <arg-string> <token-xpath expression="@src-dn"/> </arg-string> </do-append-xml-text> <do-append-xml-element expression="operation-data/return-to-me" name="destdn"/> <do-append-xml-text expression="operation-data/return-to-me/destdn"> <arg-string> <token-attr name="UserPrincipalName"/> </arg-string> </do-append-xml-text> <do-append-xml-element expression="operation-data/return-to-me" name="emplID"/> <do-append-xml-text expression="operation-data/return-to-me/emplID"> <arg-string> <token-attr name="workforceID"/> </arg-string> </do-append-xml-text> <do-append-xml-element expression="operation-data/return-to-me" name="FullName"/> <do-append-xml-text expression="operation-data/return-to-me/FullName"> <arg-string> <token-src-attr name="Given Name"/> <token-text xml:space="preserve"> </token-text> <token-src-attr name="Surname"/> </arg-string> </do-append-xml-text> <do-set-src-attr-value name="ndusO365Status"> <arg-value type="string"> <token-text xml:space="preserve">Account Pending</token-text> </arg-value> </do-set-src-attr-value> </actions> </rule>
The conditions of the rule can be pretty much whatever each organization needs to perform the monitoring of specific classes, event types, and even target attribute changes. This example is specifically for add events that target the Office365 MSolUser class.
It is important to remember that the Output Transform policy set is AFTER the transaction has passed through the Schema Mapping so all references to classes and attributes in this rule must target the destination application’s class and attribute names and NOT the eDir classes or attributes.
After whatever desired conditions have been specified, it is time to dig into the actions, i.e. where the magic happens.
The IF condition here is very critical and should be replicated to every rule where this approach is to be used. This action checks to see if an “operation-data” element already exists as part of the XML document, i.e. the current transaction. If there is not an “operation-data” element/node in the XML document, it adds one. This is important because if you create multiple “operation-data” elements it could cause issues with the document processing as only a single node is expected.
The next line, “set operation property(“parent-node-1″,”status”)“, is equally as important and should also be replicated to any rule attempting this approach. This action essentially flags the data that follows as data to be attached to the transaction status that is returned to the driver at the end of the transaction. Without this command, the data below would be lost when the driver submits the actual transaction to the target application.
Similarly, the next line should be replicated as the previous lines to any drivers that look to utilize this approach. While this line is less critical than the others, it does allow for greater organization of your code/values in the XML document it produces.
The “append XML element(“return-to-me”,”operation-data”)” command allows you to create a new node in the XML called “return-to-me” where you can append any other nodes/values that you would like to have tracked. Alternatively, you could skip this line and just create multiple node/value pairs under the “operation-data” parent node, but this approach has proven effective and easy to read, so it is my recommended approach.
From here, the next lines of code are configurable based on each driver’s purpose and each organization’s specific needs and desired results. However, it is important to note that these values are defined in key pairs, the first line defines the element name and the second defines the element value that is associated with the element name.
“append XML element(“command”, “operation-data/return-to-me”)” creates an element named “command” that goes under the “return-to-me” element which is under the “operation-data” element of our XML document.
<operation-data> <return-to-me> <command></command> </return-me> </operation-data>
“append XML text(“operation-data/return-to-me/command”,”addUser”)” adds the text “addUser” to the newly created XML element “command”
<operation-data> <return-to-me> <command>addUser</command> </return-to-me> </operation-data>
The “append XML element” command creates the new XML element in the document in the node position specified (“operation-data/return-to-me” in this example) and the “append XML text” command adds the value associated with the element in the specified node position (“operation-data/return-to-me/command” in this example).
These key pairs of nodes and values can be anything you want basically but, when specifying element/node names, try not to include spaces or many special characters like / > < @ = or other common HTML/XML characters that may cause the final XML document to be misread/interpreted by the shim. Notice in the examples here that nodes do not include spaces but multiple words are broken up with dashes (“-“).
The special character limitation does not apply for the values however and you can put anything you want into the XML document using the “append XML text” command. Looking at the included example rule, you can see that static text such as “addUser” is added, XPath is used to pull the current object’s source DN (“XPath(“@src-dn”)“), and attributes are mapped that can come from the operation data or eDir (“Attribute(“Given Name”)“).
Repeat the combo of actions to append XML element and append XML text to create the node and value pairs to satisfy any reporting.data requirements desired for the target operation/driver/connected application. There is no limit to the amount of data that can be appended to the transaction’s XML document using this method.
Important Detail Below
But, it is always recommended to include the current user’s src-dn value in the returned data for use in the post-event logic. The status data will not include the user DN making it difficult to execute commands that may target the user’s eDir object. You will be unable to query or write data to the user without the src-dn in the post-event logic otherwise.
<do-append-xml-element expression="operation-data/return-to-me" name="srcdn"/> <do-append-xml-text expression="operation-data/return-to-me/srcdn"> <arg-string> <token-xpath expression="@src-dn"/> </arg-string> </do-append-xml-text>
Once the return data has been defined, it is time to move on to the second, and final, step in the process.
Part 2: Reacting to Transaction Statuses and Returned Data
With Part 1 complete, you should now have data being returned along with the status responses of your transactions with the data specified. In most cases, the returned data is based off the operation’s overall goal like “create user”, “modify user”, “terminate user”, “rename user”, etc. so that the emails generated in this section can provide some direction to the recipients of what to look at and what the impact to the user would be as a result of any issues.
The recipients of any emails generated from this process can vary from organization to organization, from driver to driver, or even from process type to process type depending on each organization’s separation of duties or desired approach. But, in general, typically emails generated from this solution are sent to distribution lists that include multiple resources who can address issues depending on who is available at the time the issue is reported. Also, if your organization uses a ticketing system like ServiceNow that can generate tickets based on emails, this method can be used to generate tickets automatically.
Now, to react to transaction statuses and returned data based on the examples in Part 1, Part 2 requires a new rule to be created in the Input Transform policy set on the driver’s Publisher channel. Typically, I create a new policy on the Input Transform policy set to handle these events so that the rule(s) are separate from other rules and easier to manage.
This post assumes the transaction was processed through the driver’s Subscriber channel. Status messages for ALL Subscriber events are posted back to the driver shim from the application through the Input Transform. If the logic provided in this example is placed anywhere in the driver outside of the Input Transform policy set, it will not work with Subscriber channel events.
The new rule(s) in the Input Transform should follow a similar pattern to the example rule below (again, we will break down the logic in a bit more detail below).
Which looks like this in XML:
<rule> <description>Handle Error</description> <conditions> <and> <if-xpath op="true">self::status[@level = 'error']</if-xpath> <if-xpath op="true">self::status/operation-data/return-to-me/command='addUser'</if-xpath> </and> <and> <if-xpath op="true">self::status[@level = 'error']</if-xpath> <if-xpath op="true">self::status/operation-data/return-to-me/command='modifyUser'</if-xpath> </and> </conditions> <actions> <do-set-local-variable name="srcdn" scope="policy"> <arg-string> <token-xpath expression="self::status/operation-data/return-to-me/srcdn"/> </arg-string> </do-set-local-variable> <do-set-local-variable name="lv_FullName" scope="policy"> <arg-string> <token-xpath expression="self::status/operation-data/return-to-me/FullName"/> </arg-string> </do-set-local-variable> <do-set-local-variable name="lv_Message" scope="policy"> <arg-string> <token-xpath expression="self::status/text()"/> </arg-string> </do-set-local-variable> <do-if> <arg-conditions> <and> <if-local-variable mode="nocase" name="lv_Message" op="not-equal"/> </and> </arg-conditions> <arg-actions> <do-send-email-from-template disabled="true" notification-dn="Security\Default Notification Collection" template-dn="Security\Default Notification Collection\ndusDriverRetryNotification"> <arg-string name="to"> <token-global-variable name="gcv_O365EmailNotification"/> </arg-string> <arg-string name="userFullName"> <token-local-variable name="lv_FullName"/> </arg-string> <arg-string name="userCN"> <token-local-variable name="srcdn"/> </arg-string> <arg-string name="driverName"> <token-text xml:space="preserve">O365</token-text> </arg-string> <arg-string name="driverState"> <token-text xml:space="preserve">Error</token-text> </arg-string> <arg-string name="driverMessage"> <token-local-variable name="lv_Message"/> </arg-string> </do-send-email-from-template> </arg-actions> <arg-actions/> </do-if> </actions> </rule>
In this example, the rule checks to see if the transaction status is an error (“self::status[@level = ‘error’]”). This looks at the <status> response from the application and checks the “level=” property for a value of “error”.
The status level value can be “error”, “success”, “retry”, “warning”, and “fatal”. You can check for any of these status levels using the XPath expression “self::status[@level='<desired status to check for>’]” command.
Also, in this example, there are other conditions to check for a specific “command” value in the “return-to-me” data.
“if XPath expression true “self::status/operation-data/return-to-me/command=’addUser'””
Notice how the XPath expression provides the exact node path to find the expected value (“operation-data/return-to-me/command”). This path should match the path built in Part 1 using the “append XML element” commands. If the node path does not exist in the path specified then the value cannot be tested and the condition will always return as FALSE (or NOT TRUE).
Additional conditions can be defined to fit your organization’s exact needs. Typically, multiple status levels are not combined into a single rule as most organizations want different messages sent and/or messages sent to different recipients in the event of different status levels. For example, errors may need to be sent to IDM admins for review while successes trigger emails to end-users and warnings are ignored/do not result in messages sent to anyone.
However you or your organization chooses to define those conditions, once you have them defined, you need to process that XML data and do something with it. The example here simply triggers an email notification template to be populated with data and sent to admins for review. Other use cases for this solution include updating eDir data, adding roles & resources, and even vetoing events stuck in retry loops.
The key to whatever you want to do with the data is knowing how to access the information in the XML data attached to the status response. Generally, it is recommended to capture the returned data and store it in local variables at the beginning of the rule’s actions so the variables can be referenced more quickly and easier in later actions.
“set local variable(“lv_FullName”, scope=”policy”, XPath(“self::status/operation-data/return-to-me/FullName”))” creates a new local variable called “lv_FullName” and assigns it the value of the text found in the XML node path of “operation-data/return-to-me/FullName”. If the node is not found or blank, then the variable is assigned a value of ” “.
The “lv_” in the variable name lv_FullName is short for Local Variable, a naming construct that makes it easy to distinguish local variables from attribute names in code that I like to use. Local variables can be named pretty much anything you like but I prefer the “lv_” prefix for local variables and “gcv_” for global configuration variables.
Once you have your values mapped to local variables then it is pretty easy to execute just about any standard command that a driver is capable of using that data. Of course, you are not required to map the return data to local variables and can reference it directly in other actions using the similar XPath mapping technique.
Important Detail Below
When trying to write data or query the user in eDir as part of the post-event actions, you cannot use the “Current object” selection of the “Select object” property of your rule. This is the post-event status so there is NO “Current object”. You must choose the “DN” option and then specify the target object’s full eDir DN, which is why it is always recommended to include the src-dn as part of the return data when using this solution.
Even More Important Details
Even though this is post-event actions based on an event from the Subscriber channel, this logic is still executed on the Publisher channel. As such, anything that needs to target eDir is considered the Destination while anything targeting the connection application is considered the Source, just like any normal event on the Publisher channel. It is a common misconception that since these actions are based on the status of a Subscriber channel event that the targets are the same as the Subscriber channel. This rule acts on the Publisher channel and as such the targets are consistent with all other transactions on that channel.
If I wanted to update an user object in eDir after a successful event I would need an action similar to this:
“add destination attribute value(“mail”, direct=”true”, dn(XPath(“self::status/operation-data/return-to-me/srcdn”)), “email@example.com”)”
Since this is on the Input Transform of the Publisher channel, I have to do an “add destination attribute value” to target eDir.
In this example, I am targeting the “mail” attribute.
Next, the action is set to do a direct write to the data store (direct=”true”) and this is because there is no current operation that this action can be appended to. This is based off a status response we need to write to eDir directly if we want this attribute to be updated.
And, as mentioned above, since this is a status message and there is no Current object, the action chooses to provide the DN of the target object and maps that to the object’s source DN that was included in the return-to-me data under the “srcdn” XML element/node; “dn(XPath(“self::status/operation-data/return-to-me/srcdn”))“.
And lastly, we provide the value to be added to the target user’s target attribute. In this case we are adding the email address “firstname.lastname@example.org” to the target user’s “mail” attribute in eDirectory.
Now, this solution can be useful even outside of the holidays and during times when organizations do not typically have reduced staffing levels. It is a handy way to ensure your IDM solution only takes certain actions AFTER an account has been created or otherwise modified in a connected system instead of doing it as part of the initial transaction without knowing if event was actually successful (mainly because the event hasn’t been completed yet). But, during the holidays where people are taking vacation days and the office may be closed during normal business hours, this approach is very helpful in keeping an eye on the IDM data and events even if the person/team normally responsible are not available.