• Home
        • Database Development

          Database development is designing, creating a database or data model, and analyzing requirements and their intents as raw data.

          Learn More
        • Architecture & Design

          Software architecture refers to the fundamental structures of a software system and the discipline of creating such structures and systems.

          Learn More
        • Programming

          Computer programming is the process of performing a particular computation or more generally, accomplishing a specific result.

          Learn More
        • Cloud Computing

          Cloud computing is the on-demand availability of computer system resources, especially data storage and computing power.

          Learn More
        • ETL Development

          ETL provides the foundation for data analytics and machine learning workstreams. Through a series of business rules, ETL cleanses and organizes data.

          Learn More
        • Data Visualization & Reports

          Data and information visualization is an interdisciplinary field that deals with the graphic representation of data and information.

          Learn More
  • Blog
  • Contact

Compress Dynamic Files Using 7-Zip in SSIS

Abstract

This blog post is in response to a request/comment from Thato in this previous article. Hence, I have taken much of the logic used in the aforementioned article and changed it from static to dynamic file compression. Thus, I am still using SQL Server Integration Services’ Execute Process Task to compress dynamic files into a .rar format using 7-Zip.

Requirements

Article
Let’s begin by launching Microsoft Visual Studio and creating a new Integration Services Project which is located under Business Intelligence Projects category. (If you are using SQL Server 2012 or later, you will have to launch SQL Server Data Tools – SSDT).
After you have assigned a project name, let’s add some variables that will be used to store the dynamic content. I set up the variables as follows:
""

  • var_FileName (is of type String and stores file name and extension)
  • var_WorkingDIR (is of type String and stores path for working directory)
  • var_NewFileName (is of type String and stores a derived archive file name)
  • var_Exec (is of type String and stores the path to the executable 7-Zip file)

Now let’s click and drag the ForEach Loop Container (FLC) into the Control Flow pane from the toolbox (SSIS Toolbox in SSDT).
Right-click FLC and click on “Edit”. Under the Collection tab, ensure that Enumerator is Foreach File Enumerator. Set up the Directory as an expression and map it to the var_WorkingDIR variable. Lastly, make sure that Name and extension are checked under the Retrieve file name group box. In summary, the Collection tab should be configured as below:
""
Still on FLC, under the Variable Mappings tab map variable var_FileName to the first index 0 as shown below:
""
The next step is to drag a Script Task (ST) into the FLC. The ST will be used to initialize variable var_NewFileName. Under the Script tab of the ST ensure that you have configured the read/write variables as shown below:
""
Still on ST, click Edit Script. Reference the System namespace as shown below:

The rest of the code in the Main class should be as follows:
""
The final step is to drag and drop the Execute Process Task (EPT) into the Control Flow pane from the toolbox (SSIS Toolbox in SSDT).
Right-click EPT and click on “Edit”
Under the General Tab, you can assign relevant Names & Descriptions. In this case, I have decided to name the task as “EPT – Compress Dynamic Files” whilst the Description remains unchanged.
In the Process Tab, the EPT properties must be set as follows:

  • RequiredFullFileName : True
  • FailTaskIfReturnCodeIsNotSuccessValue : True
  • SuccessValue : 0
  • Timeout : 0
  • WindowStyle : Hidden

Still, on the EPT, we now need to configure expressions. Thus the expression tab should look as follows:
""

Basically, some of the noticeable things we have done above are to hide the exe window that appears when the file is being compressed. The Executable and WorkingDirectory properties are set via variables. The Arguments property is also set using variables and we have further hard-coded the extension of the compressed file to be that of type rar. Luckily 7-Zip supports various file compression types, so you can change the rar extension to an extension of your choice. Of course, depending on the extension you have provided the size of the new compressed file will vary.
PS: You can download a copy of this package here.
It’s that simple.
Till next time folks, cheers.
Sifiso.
CodeProject

Loading

Sifiso

October 13, 2022
Sifiso is Data Architect and Technical Lead at SELECT SIFISO – a technology consulting firm focusing on cloud migrations, data ingestion, DevOps, reporting and analytics. Sifiso has over 15 years of across private and public business sectors, helping businesses implement Microsoft, AWS and open-source technology solutions. He is the member of the Johannesburg SQL User Group and also hold a Master’s Degree in MCom IT Management from the University of Johannesburg.

Meet Our Experts

We are proud to have a team of experts who are passionate about delivering the best possible solutions to you. Our team members are highly skilled and experienced in a diverse range of IT technologies, and are committed to staying up-to-date with the latest industry trends and best practices to deliver you the best results.

0 Comments

Submit a Comment

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.

Join Our Newsletter

Subscribe to get our latest and best thinking on the most definitive workforce topics affecting HR leaders and organizations today.