Agile BI Community

Todd McDermid (a frequent poster on the MSDN SSIS forums, and creator of the Kimball SCD component) and I have started a new project on CodePlex along with Todd McDermid, the SSIS Community Tasks and Components (ssisCTC) project. This project is intended to be a umbrella project for custom SSIS tasks and components that are released as open ...

I'm doing a couple of chalk talks at the Microsoft BI Conference, occurring in Seattle, WA on October 6-8. The chalk talks are a different format than the typical presentation: smaller audience, no (or very few) slides, and more conversational in format. I'm doing one on unit testing for SSIS, and another on building custom components for SSIS. I ...

A question that comes up occasionally is how to handle errors that occur on the OLE DB Destination in the data flow. Errors that occur when using the OLE DB Destination to insert into a table are usually caused by primary key, foreign key, or check constraints violations. When one of these errors occurs, the default behavior of the OLE DB ...

A number of the SSIS MVPs were asked to contribute a white paper to MSDN a few months ago, and those articles were published recently. You'll notice a common thread in most of the topics: Considerations for High Volume ETL Using SQL Server Integration Services by Pat Martin Reusing Connections with Data Sources and Configurations by ...

I came across a question about this the other day, and thought I'd share the answer here, as there doesn't seem to be much information out there about it.  The questioner wanted to know how to read a portion of an XML document into a variable. Fortunately, the XML Task allows you to do this by running an XPath query. To configure the task, ...

I'm going to be presenting a few sessions at the upcoming SSWUG Virtual Business Intelligence Conference. It's occurring from September 24-26, 2008. It has a very impressive list of speakers (I'm really not sure how I made it on the list - it's quite an honor). I'll be presenting on configurations in SSIS, unit testing SSIS (one of my favorite ...

A common question on the SSIS forums is how to continue a loop after an error occurs. For example, you might have a For Each Loop container that iterates through a set of files in a folder. If one file is invalid, you want to continue processing the remaining files. Fortunately, there is a simple pattern that allows this to be accomplished. I ...

Douglas Laudenschlager has posted some good information about the considerations you need to make if you are running SSIS on a 64-bit box. This is additional information to what is currently in Books Online, and it’s well worth looking at. Considering that a lot of developers are developing on x86 machines, and deploying to x64 servers, it’s even ...

To finish up the topic that I started in my previous post, I wanted to post a more complex example of running total calculations. This is based off a request on the MSDN forums, for a running total calculation that would “slide” along as additional rows were processed. In the poster’s example, the running total needed to reflect the total for the ...

Running totals are a common calculation for reporting, and are occasionally needed in ETL processes to populate aggregate tables. One way to accomplish this in SSIS, if you are retrieving data from a relational database, is to do the running total in the SELECT statement. There are a number of resources on the Internet that illustrate how to ...