Monday, February 18, 2013

Secrets of the Metaverse Part 3

Parts 1-5:

    1. What is the Metaverse?
    2. How is the Metaverse data stored?
    3. Is there a limit to how many Metaverse attributes I can have?
    4. Has access to the metaverse gotten faster with recent releases?
    5. How do I safely query the metaverse?

    Many times people wonder how many attributes they can create in the Metaverse Designer tool.

    The answer is confusing because ... it depends.

    Per my calculations there is a hard limit for single-valued, non-reference attributes of 502. Now to show you my work (in school my math teachers always insisted that I show my work).

    Remember from Part 2 that the Metaverse consists of 5 tables:

    When you create a new single-valued, non-reference attribute using the Metaverse Designer tool FIM modifies not just data but the table structure of three of these tables: mms_metaverse, mms_metaverse_lineagedate, and mms_metaverse_lineageguid. FIM will add a column to each of these tables. The mms_metaverse table will get a column of a datatype  based on what has been selected as the attribute data type:

    Metaverse DataType

    SQL Data Type

    Min Size Max Size Comments

    String (Indexable)

    nvarchar(448)

    2 898  

    String (non-Indexable)

    nvarchar(max)

      2 GB If the data is small (less than 4000 bytes or whatever limit is set) and doesn't cause the row to exceed the 8060 byte limit it will stored in the row, otherwise only pointers will be stored in the row and the data will be stored in its own set of pages (off-row).

    Binary(indexable)

    varbinary(900)

    2 902  

    Binary(non-indexable)

    varbinary(max)

      2 GB see the note on string non-indexable

    Boolean

    bit

    1/8 1

    If there are between 1-8 bit columns in the table 1 byte of storage is consumed, if 9-16 bits then 2 bytes, if 17-24 then 3 bytes and so on

    Number

    BigInt

    8 8  

    In the mms_metaverse_lineagedate table FIM adds a new column of type DateTime which takes 8 bytes. In the mms_metaverse_lineageguid table FIM adds a column of type UniqueIdentifier which takes 16 bytes.

    A row in SQL Server 2000 and beyond can only hold 8060 bytes. With SQL 2005 and beyond there is a feature called row overflow that allows for the variable length columns (like varchar) to overflow to other pages. However that doesn't apply to fixed length data types like BigInt, DateTime and UniqueIdentifier.

    So the mms_metaverse_lineageguid has two columns (row_key a bigInt and object_id a unique identifier so 24 bytes) and then one column of type UniqueIdentifier for every single-valued non-reference attribute. 8060-24 = 8036 and 8036/16 = 502.25, which rounds down to 502.

    So while there may be other limits that may depend on the datatype selected, this is one limit that cannot be escaped.

    Friday, February 15, 2013

    Secrets of the Metaverse Part 2

    Parts 1-5:

      1. What is the Metaverse?
      2. How is the Metaverse data stored?
      3. Is there a limit to how many Metaverse attributes I can have?
      4. Has access to the metaverse gotten faster with recent releases?
      5. How do I safely query the metaverse?

      Where and how is the Metaverse data stored?

      Before I get into that I must caution you that modifying data directly will put you in a position that is unsupported by Microsoft. Even querying the data is something of a touchy issue (see Part 5).

      The Metaverse consists of 5 tables in the FIM Synchronization Service Database:

      Table Comment
      mms_metaverse Every object in the metaverse has a row in this table. Single-Valued non-reference attributes are stored in this table
      mms_metaverse_lineagedate This table has a DateTime column of the same name of every attribute column in the mms_metaverse table (in other words -- single-valued non-reference attributes).
      mms_metaverse_lineageguid This table has a UniqueIdentifier column of the same name of every attribute column in the mms_metaverse table (in other words -- single-valued non-reference attributes).

      mms_mv_link

      Reference attributes (both single valued and multi-valued) are stored in this table in an Entity Attribute Value format. The references are kept as uniqueIdentifiers
      mms_metaverse_multivalue Non-Reference multi-valued attributes are stored in this table in an Entity Attribute Value format (with a column for each of the possible data types)

      Thursday, February 14, 2013

      Secrets of the Metaverse Part 1

      Many FIMsters wonder about the Metaverse and how works, how the data is stored. In this series I will reveal the secrets of the Metaverse. Parts 1-5 (links live but post yet to come)
      1. What is the Metaverse?
      2. How is the Metaverse data stored?
      3. Is there a limit to how many Metaverse attributes I can have?
      4. Has access to the metaverse gotten faster with recent releases?
      5. How do I safely query the metaverse?
      6. Added (Aug 5 2015): How Many Metaverse Attributes can I have?
      Forefront Identity Manager 2010 R2 SP1 (and its predecessors) can be classified as a MetaDirectory based Identity Management Solution. A MetaDirectory collects, aggregates, and stores data from various directories and data sources, such as Active Directory and your HR database.
      The Metaverse is the heart of FIM's MetaDirectory. As an implementer of FIM you customize the data model, you decide what object types and attributes you need.

      Updated Vote: Top 5 Deprecated Features of FIM 2010 R2 SP1

      Here is an update on the impact of the newly deprecated features: The big change is that XMA has caught up to Multi-Mastery and is tied for first.

      image

      Massive FIM and AD LDS project at DPDHL

      Watch the presentation that James Booth (who worked with us on the project) and Joe Gasowski (DPDHL) gave at the Redmond Identity Summit 2013 about our project at DHL to replace the DPDHL Sun One Directory and deploy FIM to replace both CriticalPath and a home-grown admin portal.