My quest to discover and teach Best Practices in Identity Management especially with Microsoft FIM / ILM -- David Lundell of Identity Managed
Technical overview whitepaper on FIM 2010 (download)
Brad and I spent many long hours writing this! Glad to see it come out in the long form. Thanks to the product group for the opportunity and the Brjann, Mark, and Markus for reviewing and editing it.
Microsoft has honored the efforts of our Identity and Secure Access Management Practice by making us a finalist for 2010 Partner of the Year, Core Infrastructure Solutions, Server Platform, as a result of our work with FIM, ILM, AD, AD FS and AD CS.
Recently, I discovered that under certain conditions the removal of a dependent sync rule could cause the disconnection of objects in AD or other connected data sources. So I had to investigate the inner workings of dependent Sync Rules to uncover this mystery and fix it.
FIM allows us to create dependent Sync Rules. First let me explain the what and then a little why. Then allow me to explain a bug that I discovered and how to work around it.
A Dependent Sync Rule is an outbound sync rule, or and inbound and outbound sync rule (with only the outbound side working) that depends on and inherits (at the time of creation) from its parent or base sync rule.
A base Sync Rule is one that doesn’t depend on any other sync rule.
A dependent sync rule can depend on another dependent sync rule.
A dependent sync rule cannot be applied to a resource until its parent sync rule has been applied
When you create a sync rule as a dependent sync rule it inherits a number of attributes from the parent
Let’s break it down:
The resource Type is obviously sync rule
Create External System Resource, Create FIM Resource, and Disconnect External System Resource are all set to false because this is a dependent sync rule. Back in RC 1 Disconnect External System Resource inherited the value of the parent which leads to odd behavior I will describe further on in the post. The attributes and their corresponding actions are the province of base sync rules.
Data Flow Direction (1 for outbound or 2 for in and out), External System, External System Resource Type, FIM Resource Type, and Relationship criteria are inherited from the parent.
Workflow Parameters and persistent flow can be set as well as the DisplayName and the Description. Initial flow can’t be set as that is also the province of base sync rules.
Why implement dependent sync rules? Flow additional attributes in a situational fashion but only if some base sync rule is already applied.
One danger can occur. If you create a base sync rule, set the Disconnect External System Resource to true and then change it to a dependent sync rule, the disconnect remains true and you have no way in the UI to change it since the scoping and relationship tabs are hidden for dependent sync rules. So the bug is either that you can’t change it in the UI or that the UI or service should be changing it to false if the rule becomes dependent.
If this setting is set to true then it might disconnect the resource when the dependent sync rule is removed.
This can also happen if you load a sync rule through the web service as dependent and set the Disconnect External System Resource to true.
How did it get this way? Well if you were an RDP or TAP customer and you had RC 1 and then upgraded your FIM Service database it could be that way. Or if you export the FIM sync rule from one system and it was like that and then went to another it would load in with it set to true
How to find out if you have this?
Assuming you have already imported and sync’d from the FIM MA then In the Sync Service Manager, go to metaverse search. Search for Sync rules. Search for those with the DisconnectConnectedSystemObject set to true and dependency is present. If anything turns up then you have this issue.
To solve it you can use the UI or this PowerShell script against the web service.
To use the UI you open the dependent sync rule change it to not have a dependency and then change it back to depend on the one that it had before. That seems to tell the UI to set Disconnect External System Resource to false
Initially your dependent sync rule looks like this:
So change it to look like this (but don’t click OK and submit – not yet anyway)
Then change it back
Then click OK and you should see this and nothing else getting changed
Then click Submit.
Or use PowerShell on your next import and sync from FIM your sync rule will be fixed.
. C:\FIMTasks\FIMHelper\FIMPowerShell.ps1
# includes the helper functions from: http://technet.microsoft.com/en-us/library/ff720152(WS.10).aspx
# and the GenerateFilter from http://technet.microsoft.com/en-us/library/ff720142(WS.10).aspx
# and GetAttributeValueFromResource from http://technet.microsoft.com/en-us/library/ff720158(WS.10).aspx
# take the above helper functions and create a file called FIMPowerShell.ps1 and put them in it
# Purpose of this script is to Identify sync rules with dependencies
# where the DisconnectConnectedSystemObject=true and set it to false
# step 1 find Sync rules with dependencies and DisconnectConnectedSystemObject=true
# step 2 Modify the DisconnectConnectedSystemObject=false for each of these
# step 3 commit to FIM
function QueryResourceOnlyBase
{
PARAM($Filter, $Uri = $DefaultUri)
END
{
$resources = Export-FIMConfig -CustomConfig $Filter -Uri $Uri -OnlyBaseResources
$resources
}
}
# step 1 find Sync rules with dependencies and DisconnectConnectedSystemObject=true
$uri = "http://localhost:5725/ResourceManagementService"
$exportedSyncRules = QueryResourceOnlyBase -Filter "/SynchronizationRule[DisconnectConnectedSystemObject=true
and Dependency = /SynchronizationRule ]" -Uri $Uri
Write-Host ("The script has exported " + $exportedSyncRules.Count + " sync rules based on the filter: " + $query)
# step 2 Modify the DisconnectConnectedSystemObject=false for each of these
$importList = @()
foreach ($syncRule in $exportedSyncRules)
{
$importObject = ModifyImportObject $syncRule.ResourceManagementObject.ObjectIdentifier "SynchronizationRule"
SetSingleValue $importObject "DisconnectConnectedSystemObject" "false" 1
$importList += $importObject
}
# step 3 commit to FIM
if ($importList.Count -gt 0 )
{
$undoneChanges = $importList | Import-FIMConfig -Uri $Uri
# Use ConvertFrom-FIMResource cmdlet to convert contents of $undoneChanges to the file "undone.xml".
if ($undoneChanges.Count -gt 0)
{
Write-Host "There are " + $undoneChanges.Count + " sync rules that failed to update"
Write-Host "The undone imports are written to undone.xml"
$undoneChanges | ConvertFrom-FIMResource -file undone.xml
}
Write-Host "Import complete"
}
else
{
Write-Host "Nothing to import"
}
Lessons learned:
1) Run the Do a FIM MA account configuration quick test script.
2) Always refresh the schema of the FIM MA using the real FIM MA Service Account which we usually call svc-FIMMA.
Scenario:
You have just modified the schema of FIM Service by creating a new Boolean attribute and have bound it to the user resource type. You refresh the FIM Schema, select the new attribute setup a direct export attribute flow from the corresponding Boolean metaverse attribute to the FIM MA attribute. You sync and the only pending export is to this attribute, and then when you run the export to FIM MA you get:
failed-modification-via-web-services
Then when you click on details:
There is an error executing a web service object modification request.
Type: System.NullReferenceExceptionMessage: Object reference not set to an instance of an object.
Stack Trace: at MIIS.ManagementAgent.RavenMA.DoAttributeLevelExport(DataSourceObject dsObject, String objClass, UninitializedResource resource)
at MIIS.ManagementAgent.RavenMA.ExportObjectModification(DataSourceObject dsObject, SchemaManager schemaManager)
at MIIS.ManagementAgent.RavenMA.Export(DataSourceObject dsObject)Inner Exception:
I have encountered this twice and Brad once. The first time (1 month ago) I discovered that the FIM MA was set to use the svc-fimws (this is the credentials used to run the FIM Service) user account instead of the svc-fimma (the account that actually belongs to the Synchronization Engine Set and therefore has all of the permissions to modify objects in the portal and bypasses the authn and authz workflows and is sometimes referred to as the Built-in Synchronization account). So I changed the FIM MA to use the svc-fimma account and when prompted clicked yes Refresh Schema. The Do a FIM MA account configuration quick test script had told me that the two weren’t the same (go FIM Scriptbox!)
Which means that when I ran the refresh schema the first time it was under the svc-fimws account not the svc-fimma account.
The second time we refreshed the schema using the credentials of an admin user as opposed to the svc-fimma credentials. I tried refreshing the schema with the svc-fimma account but of course it said no changes. I tried stopping and starting all of the services. I turned on messaging logging (this isn’t even sending messages it is dying on the MA without even talking to the web service). So after carefully studying my memory (foolish me I didn’t blog it or make a note) and the request history on the other system I remembered what the root problem and solution had been. But I had already checked if the MA was using the true Built-in Synchronization Account using the Do a FIM MA account configuration quick test script . So then I thought what if I create the exact same problem I had last time and then fix it. So I changed the FIM MA to use an admin user account and when prompted clicked yes Refresh Schema. Then I changed the FIM MA to use the svc-fimma account and when prompted clicked yes Refresh Schema. Then my export worked!
Lessons learned:
1) Run the Do a FIM MA account configuration quick test script.
2) Always refresh the schema of the FIM MA using the real FIM MA Service Account which we usually call svc-FIMMA. (Anyone from Microsoft want to give an official statement of best practice on that?)
I plan to repro this in a test environment to ensure this is the problem.
A little while ago I encountered some rather strange behavior of a Set vs. the XPATH query in FIM 2010.
Using the Export-FIMConfig with the -onlyBaseResources -CustomConfig switches I run the following query to see if there are any users without a DisplayName
/Person[not(starts-with(DisplayName,''))]
It showed 20
So then I created a set, called “~ People with no displayname”, with that as the custom filter. I checked it doesn't violate any of the limitations listed in the Business Policy Modeling doc (which I must say is a pretty good doc)
Then when I look at the Set and click view members on the criteria tab it shows 20 users. So far so good
But when I go to Search for users and it Resource ID in “~ People with no displayname” it shows me over 10,000
Indeed using the commandlet to run this query /Person[ObjectID=/Set[DisplayName='~ People with no displayname']/ComputedMember]") I get over 10,000
Jeremy made a suggestion:
/Person[not(starts-with(DisplayName,'%'))]
Sure enough it works both as the SET filter and as the XPATH query and showed 20 records.
So to test a string for null, use:
not(starts-wth(Attribute,'%'))
in the XPATH predicate.
Why does this work?
The starts-with function works just like using the LIKE operator in T-SQL with an implied % at the end. not(starts-wth() does a NOT LIKE '%' with the implied %. Since % will match anything as long as it is not null this effectively tests for null.
All of the wildcards available in the LIKE operator work
I have tested it using the single wildcard character _ as well as ranges like [a-c] and other more complex patterns.
This also means that you can effectively do a contains in a set or group filter by doing:
starts-wth(attribute,'%searchvalue')
That’s right just prefix your searchvalue with % and what happens behind the scenes is a LIKE '%searchvalue%' which will find searchvalue anywhere in the string.
Warning about View Members on a set (possibly groups too)
Apparently when I click on view members on the Set’s Criteria tab it runs the XPATH query right then. But when you save the set with its new filter it runs the query in a slightly different fashion by first persisting the SET criteria to the database and then reloading the criteria and running the query and then persisting the membership results in the database to speed up look ups. (Naturally, the penalty is every create, delete, modify, add and remove request requires each set to be examined for impact and possible recalculation). So when you use the filter do do /Person[not(starts-with(DisplayName, '%'))] it stores that piece of the criteria as the literal string %% and the operator as LIKE. But when you use /Person[not(starts-with(DisplayName, ''))] it seems to delete that piece of the criteria effectively making it a set of all Persons. If you are implementing a ROPU (pronounced Rope You – Run on Policy Update) enabled workflow and this kind of thing happens to you it can me that a workflow is being applied to 10,000 objects instead of 20.
Jun 10, 2010
1:00 p.m. Eastern / 10:00 a.m. Pacific (60 minutes)
Christopher Yeich - Editor, Strategic Content - Ziff Davis Enterprise
David Lundell - Identity Management Practice Director, Ensynch | Microsoft Identity Management MVP
Jonathan Sander - IAM and Security Analyst - Quest Software
Has your business experienced identity theft, with unauthorized access to your systems, data, and/or trade secrets?
Have you lost business because your customers and/or employees didn’t have access when needed?
How much time have you wasted in producing compliance/regulatory reports for various auditors?
These are all real-life situations that business and IT leaders like you are experiencing every day. Breaches lead to millions—sometimes billions—in lost monies every year. Additionally, there's also confusion, frustration, and lost productivity that organizations deal with every day as they fight to manage appropriate access to information and tools that employees, business partners, and customers actually need.
Join Microsoft Identity MVP David Lundell of Ensynch, and Jonathan Sander, IAM and Security Analyst of Quest Software, for a candid presentation that uncovers ways you can protect and accelerate your business—as well as save money—with identity and secure access management (ISAM).
Topics of discussion will include:
Searching an entire database for a Guid or Unique Identifier can be a bit of a tricky proposition. However a little bit of using T-SQL to generate T-SQL and viola
DECLARE @GUIDHunted nvarchar(60)
SET @GUIDHunted = '0A24EC0C-65EE-4519-89DF-ABD3DD24F7EF'
SELECT *, 'UNION ALL SELECT ''' + s.name + '.' + ao.name + ''', count(*) FROM '
+ s.name +'.[' + ao.name + '] WHERE ' + ac.name + ' = ''' + @GuidHunted + ''''
FROM sys.all_columns ac
JOIN sys.all_objects ao
ON ac.[object_id] = ao.[object_id]
JOIN sys.schemas s
ON ao.[schema_id] = s.[schema_id]
where user_type_id = 36 -- UniqueIdentifier
and s.name != 'sys'
Here is the output result for a fictional database (just copy the results into a new query window, delete the first UNION ALL and execute).
UNION ALL SELECT 'myschema.Objects' , count(*) FROM myschema.Objects WHERE ObjectID = '0A24EC0C-65EE-4519-89DF-ABD3DD24F7EF'
UNION ALL SELECT 'myschema.Objects2' , count(*) FROM myschema.Objects2 WHERE ObjectID = '0A24EC0C-65EE-4519-89DF-ABD3DD24F7EF'
This assumes that programmers weren’t storing guids in nvarchar types or that the programmers didn’t create other user types using unique identifiers
