BIML Annotations


BIML offers a great method to classify and manage code through annotations.  Basically, an annotation is just a tag that provides the ability to include text into our BIML code.  No big deal right?  Well perhaps not, however, if we think about how we can use this feature a little deeper, we’ll see there’s some very powerful capability just under the covers. 

Consider for example, using BIML Script to define a large number of packages using a standard template from different source systems.  We may want to ‘tag’ each package with a tag that specifies the system.  We could a that tag to specify the system within each package and then use the value of the tag later in our script (say for example creating a master package that groups all the child packages of one system).

Lets look at a simple example which follows this scenario. In the script below, I generate 4 packages (which don’t do anything) and have an annotation called ENV in each of them.  The annotation has the Value ‘Test’.  You’ll also notice that I’ve specified this script as tier 1 (meaning that its executed first).

<#@ template language="C#" tier="1" #>
<# string _env="Test"; #>
  <Biml xmlns="http://schemas.varigence.com/biml.xsd">
    <Packages>
      <# for(int i=1;i<5;i++){ #>
        <Package Name="Package<#= i.ToString() #>" ConstraintMode="Linear">
          <Annotations>
            <Annotation Tag="ENV"><#= _env #></Annotation>
          </Annotations>
        </Package>
        <#} #>
    </Packages>
  </Biml>

The annotation tag (‘ENV’) is designed to add context to the package when the Biml Script is compiled/compiled.  You can’t see this if your using BIDS helper however I’ll show the expanded BIML is shown below.  Notice that each package has the tag ENV.

<Biml xmlns="http://schemas.varigence.com/biml.xsd">
	<Packages>
	<Package Name="Package1" ConstraintMode="Linear">
		<Annotations>
			<Annotation Tag="ENV">Test</Annotation>
            </Annotations>
	</Package>
        <Package Name="Package2" ConstraintMode="Linear">
		<Annotations>
		<Annotation Tag="ENV">Test</Annotation>
            </Annotations>
		</Package>
        		<Package Name="Package3" ConstraintMode="Linear">
			<Annotations>
				<Annotation Tag="ENV">Test</Annotation>
            </Annotations>
		</Package>
        		<Package Name="Package4" ConstraintMode="Linear">
			<Annotations>
				<Annotation Tag="ENV">Test</Annotation>
            </Annotations>
		</Package>
            </Packages>
</Biml>

Now, I can now use that tag in other BIML scripts.  So, (for example) suppose I want a master package that execute my ‘Test’ packages.  All I would have to do is iterate over each of my packages (in the solution) and test each to see if they belonged to the ‘Test’ ENV(environment).  If they did contain that ENV value, I just include them in an execute package task.  Here’s how the second script looks;

 
<#@ template language="C#" tier="2" #>
  <Biml xmlns="http://schemas.varigence.com/biml.xsd">
    <Packages>
      <Package Name="MASTER" ConstraintMode="Linear">
        <Tasks>
          <# foreach (var _package in RootNode.Packages){
					   if (_package.GetTag("ENV")=="Test"){#>
          <ExecutePackage Name="Run <#= _package.Name #>">
            <Package PackageName="<#= _package.Name #>" />
          </ExecutePackage>
          <#} } #>
        </Tasks>
      </Package>
    </Packages>
  </Biml>

In the code above, you can see how I reference the a packages tag using the GetTag method.  Also note that the tier for this package is 2 so its executed after the prior script.  Once compiled, the output is a master package that executes each child (as below);

image

Of course this is a really simple example and you may suggest you could achieve something similar using the package name.  For example you could iterate over each package using the foreach (var _package in RootNode.Packages) above and reference _package.Name. 

This approach may work (of course) however, it relies on your package name fully encapsulating the functionality of all possible annotations – a process that could be very messy.  Suppose, I wanted to include creator, ‘ETL Stage’, ETL Version etc, soon my package name would be unmanageable.

If you are using BIDS helper, you may also consider using a different visual studio project for each ‘package type’, purpose and environment.  Again, this solution may work however, lets consider our original proposition.  We are using exactly the same pattern with different meta-data (that is, the environment).  Wouldn’t it make things simple to maintain this within the one solution?

This is where MIST can really help with development.  I use the same BIML script to generate multiple packages which are all held within the same solution, they are then compiled into different Visual Studio solutions.  Put simply, MIST allows me to keep everything in the one place.

Advertisements

4 thoughts on “BIML Annotations

  1. Hi Paul
    One suggestion, I think you code would be slightly more elegant with.

    x.GetTag(“ENV”) == “Test”)){#>
    <ExecutePackage Name="Run “>
    <Package PackageName="” />

    Cheers mate, Simon

  2. Not sure what happened in my last comment, it’s not showing as I typed it. I’ll try again

    x.GetTag(“ENV”) == “Test”)){#>
    <ExecutePackage Name="Run “>
    <Package PackageName="” />

    • Hi Steven,
      Im not sure if your still loosing something however I think there will be some issues with that snippet. Using ‘Run’ as the for the execute package task will throw an error (when there’s more than 1 package (becuase you cant have tasks with the same name) and there is no reference to the package name that should be executed.

      • Hi Paul. Excellent article!
        Wordpress comments aren’t BIML friendly. I was simply pointing out that you could have slightly more compact code with.

        var _package in RootNode.Packages.Where(x => x.GetTag(“ENV”) == “Test”)){

        You could then ditch the If statement

        Cheers
        Simon

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s