Where It All Started.

Where It All Started.

Life, Stock Trading, Investments, Business and Startup. Most are programming stuff.

Tag: python

Make Subfolder A Git Submodule

Ever been in a situation where a sub-folder of your git repository needs to branch out as a new repository? Here in this article I’ve tried a new way using a python module to simplify the process. This steps is also recommended by the Git core team as ways to move a sub-folder to a new clean repository (link).


We’re just clones, sir. We’re meant to be expendable.

— Sinker.

Come on let’s jump in! 🚀

Prerequisites

First of all, you must have a Git1 on your machine. Second, must have existing test git repository and Python 32 installed.

If you don’t have Git yet, you can install git from its official sources, its available on all platforms even on android. Or if you have Visual Studio3 installed, just locate it from your drive. Python can also be installed using the Visual Studio installer.

So where do we start?

This will be our initial test repository structure:

+-+ root
  |
  +-+ test-repository
  | |
  | +-+ desired-directory
  |   |
  |   +-+ contents
  | +-+ other-directory

The first step you need to do is clone the test repository by either copying it by cp command or by creating a duplicate cloned copy using git clone.

+-+ root
  |
  +-+ test-repository
  | |
  | +-+ desired-directory
  |   |
  |   +-+ contents
  | +-+ other-directory
  +-+ test-repository-copy
  | |
  | +-+ desired-directory
  |   |
  |   +-+ contents
  | +-+ other-directory

Then install this python module named git-filter-repo. Install the module using the pip utility.

pip3 install git-filter-repo

This git-filter-repo simplifies the process of filtering files, directories and history. This tool as said on its Github page falls on the same category as git-filter-branch. You can check its Github repository page for pro’s and con’s against similar tools.

Next thing we do is go into the cloned test repository and filter the directory you want (in our case its the desired-directory) to separate into a new repository.

cd test-repository-copy
git filter-repo --path desired-directory --subdirectory-filter desired-directory

This will modify the cloned directory history and delete existing content that does not match the subdirectory filter. The new structure of the directory will be like this:

+-+ root
  |
  +-+ test-repository
  | |
  | +-+ desired-directory
  |   |
  |   +-+ contents
  | +-+ other-directory
  +-+ desired-directory
  | |
  | +-+ contents

The desired-directory will now become its own repository retaining the history of files that is inside.

After moving the sub-folder to its own repository, we go back to our original test repository and delete the filtered directory.

cd test-repository
git rm -rf desired-directory

Still on the test-repository, create a new git submodule and link the filtered directory repository.

git submodule add ../desired-directory desired-directory

That’s all the steps needed, check if everything is working. Check the quick review below for a summarized setup.

Quick Review

Here are the simplified steps based on the above:

  1. Make a copy or clone the current project where the sub-folder is located.
  2. Install git-filter-repo using the command pip3 install git-filter-repo.
  3. On the cloned project folder, filter it base on the directory you want to make new repository with the command git filter-repo --path <new-path> --subdirectory-filter <filtered-directory>
  4. Go to your current project folder and delete the sub-folder using the command git rm -rf <filtered-directory>.
  5. On the current project create the sub-module using git submodule add <new-repo-url> <filtered-directory>.
  6. Check if everything is okay.

That’s all guys, always make a backup of your repository before proceeding. Anyways its on git version control system – you can go back and re-fix if there is something wrong.

Conclusion

There are many answers in the internet regarding this matter, but mostly they don’t explain what will occur when you run this command, this one I’ve personally tried it as before I was using a mono repository setup. But it became so large that its hard to maintain especially on cases of testing and checking the history.

Let me know in the comments if you have questions or queries, you can also DM me directly.

Follow me for similar article, tips, and tricks ❤.


  1. Git (/ɡɪt/) is a distributed version-control system for tracking changes in any set of files, originally designed for coordinating work among programmers cooperating on source code during software development. ↩︎
  2. Python is an interpreted, high-level and general-purpose programming language. Python’s design philosophy emphasizes code readability with its notable use of significant whitespace. Its language constructs and object-oriented approach aim to help programmers write clear, logical code for small and large-scale projects. ↩︎
  3. Microsoft Visual Studio is an integrated development environment (IDE) from Microsoft. It is used to develop computer programs, as well as websites, web apps, web services and mobile apps. Visual Studio uses Microsoft software development platforms such as Windows API, Windows Forms, Windows Presentation Foundation, Windows Store and Microsoft Silverlight. It can produce both native code and managed code. ↩︎

Gimp Automating Image Processing with Python Fu

What a large volume of adventures may be grasped within the span of his little life by him who interests his heart in everything.

— Laurence Sterne.

Hi guys, one night I decided to create an online store and sell some drop ship products. I grab some pictures from the wholesale seller and planned to customize those images (just to put some store branding).

There were a hundred images that I want to customize, by hand it would take ages so I decided to create a batch script. My first thought was to use Gimp (an open source image manipulator) and Script-Fu. After trying out what the result would be on Python-Fu console, I settled with this. It was a simple design but I was satisfied.

Here is the script that I use. Try it out on Python-Fu console and call it convert_to_poster(n), where n is the image number in the image list inside Gimp. You could for loop it for faster batch processing.

First we set the variables.

current_image = gimp.image_list()[n]
center_x = current_image.width / 2
center_y = current_image.height / 2
c_size = 600
c_size_d = c_size / 2 

Then we resize the image / canvas.

pdb.gimp_image_resize(current_image, c_size, c_size, c_size_d - center_x, c_size_d - center_y)

We create an empty background layer and just fill it with white.

bg_layer = pdb.gimp_layer_new(current_image, c_size, c_size, RGBA_IMAGE, "bg", 100, LAYER_MODE_NORMAL)
pdb.gimp_drawable_fill(bg_layer, FILL_WHITE)
pdb.gimp_image_add_layer(current_image, bg_layer, 1)

We will also set the main background and foreground color that would be use in future instruction set.

pdb.gimp_context_set_foreground("#960acc")
pdb.gimp_context_set_background("#000000")

Create branding layer and fill the rectangular region with indigo.

branding_layer = pdb.gimp_layer_new(current_image, c_size, c_size, RGBA_IMAGE, "branding", 100, LAYER_MODE_NORMAL)
pdb.gimp_image_add_layer(current_image, branding_layer, 1)

pdb.gimp_selection_none(current_image)
pdb.gimp_image_select_rectangle(current_image, CHANNEL_OP_ADD, 0, 0, c_size, 260)
pdb.gimp_drawable_edit_fill(branding_layer, FILL_FOREGROUND)

Get poster image and resize its layer to fit canvas.

poster_layer = current_image.layers[0]
pdb.gimp_layer_add_alpha(poster_layer)
pdb.gimp_layer_resize_to_image_size(poster_layer)

Select non-alpha on poster layer and grow.

pdb.gimp_image_select_item(current_image, CHANNEL_OP_REPLACE, poster_layer)
pdb.gimp_selection_grow(current_image, 3)

Create a mask and fill it with the background color that we set.

mask = pdb.gimp_layer_create_mask(branding_layer, ADD_MASK_WHITE)
pdb.gimp_layer_add_mask(branding_layer, mask)
pdb.gimp_layer_set_edit_mask(branding_layer, 1)
pdb.gimp_drawable_edit_fill(mask, FILL_BACKGROUND)

Add text node with color it white.

pdb.gimp_context_set_foreground("#ffffff")
text_layer = pdb.gimp_text_fontname(current_image, None, 10.0, 10.0, "Psyche Digital", 0, 1, 24.0, PIXELS, "SF Compact Display Heavy")

We select the main layer and add a legacy drop shadow.

pdb.gimp_image_select_item(current_image, CHANNEL_OP_REPLACE, poster_layer)
pdb.script_fu_drop_shadow(current_image, poster_layer, 10.0, 10.0, 10.0, (0, 0, 0, 255), 80, 0)

Here is the full script code:

def convert_to_poster(n):
    current_image = gimp.image_list()[n]
    center_x = current_image.width / 2
    center_y = current_image.height / 2
    c_size = 600
    c_size_d = c_size / 2 
    
    pdb.gimp_image_resize(current_image, c_size, c_size, c_size_d - center_x, c_size_d - center_y)

    bg_layer = pdb.gimp_layer_new(current_image, c_size, c_size, RGBA_IMAGE, "bg", 100, LAYER_MODE_NORMAL)
    pdb.gimp_drawable_fill(bg_layer, FILL_WHITE)
    pdb.gimp_image_add_layer(current_image, bg_layer, 1)

    pdb.gimp_context_set_foreground("#960acc")
    pdb.gimp_context_set_background("#000000")
    
    branding_layer = pdb.gimp_layer_new(current_image, c_size, c_size, RGBA_IMAGE, "branding", 100, LAYER_MODE_NORMAL)
    pdb.gimp_image_add_layer(current_image, branding_layer, 1)

    pdb.gimp_selection_none(current_image)
    pdb.gimp_image_select_rectangle(current_image, CHANNEL_OP_ADD, 0, 0, c_size, 260)
    pdb.gimp_drawable_edit_fill(branding_layer, FILL_FOREGROUND)

    poster_layer = current_image.layers[0]
    pdb.gimp_layer_add_alpha(poster_layer)
    pdb.gimp_layer_resize_to_image_size(poster_layer)

    pdb.gimp_image_select_item(current_image, CHANNEL_OP_REPLACE, poster_layer)
    pdb.gimp_selection_grow(current_image, 3)

    mask = pdb.gimp_layer_create_mask(branding_layer, ADD_MASK_WHITE)
    pdb.gimp_layer_add_mask(branding_layer, mask)
    pdb.gimp_layer_set_edit_mask(branding_layer, 1)
    pdb.gimp_drawable_edit_fill(mask, FILL_BACKGROUND)

    pdb.gimp_context_set_foreground("#ffffff")
    text_layer = pdb.gimp_text_fontname(current_image, None, 10.0, 10.0, "Psyche Digital", 0, 1, 24.0, PIXELS, "SF Compact Display Heavy")

    pdb.gimp_image_select_item(current_image, CHANNEL_OP_REPLACE, poster_layer)
    pdb.script_fu_drop_shadow(current_image, poster_layer, 10.0, 10.0, 10.0, (0, 0, 0, 255), 80, 0)

Run this code in Gimp console or in Script-Fu console.

🥰😝😜😁😎

Leave a like and subscribe for more hacks, tips, and tricks.

DevOps Series – Multiple AWS CLI Accounts

Lack of direction, not lack of time, is the problem. We all have twenty-four hour days.

— Zig Ziglar.

As a Dev Ops, I handle multiple project all at once. One problem I encountered is managing multiple AWS accounts for different startups. So how to handle multiple AWS account?

Prerequisites

Before you start doing this, I assume you already know basic command line interface whether its for Windows or from *Nix (Linux, macOS, Unix) derivatives. And also you must have a python interpreter with at least version 3 above.

Multiple, Multiple to the Nth

First, if you haven’t got the AWS CLI (Command Line Interface) install it using the command pip install --upgrade --user awscli. The command will install the awscli package in your local python dist directory.

And now we begin.

You’ll need to generate first your AWS keys and secret access key. Here are the steps.

  • Go to IAM Console
  • Go to Users in the navigation pane and select your IAM username.
  • Select Security credentials and choose Create access key.

Then, after that we will need to configure our AWS using aws configure --profile <your-profile-name>.

aws configure --profile my-profile-name

This command will need some more input from you like your AWS Key, AWS Secret, AWS Account Region and the default output format which would be JSON or TEXT.

If everything is setup properly, we will proceed with running a some sample commands. Also in order for you to use the configured AWS profile, you must always append at the end of your AWS command the --profile <your-profile-name>.

aws configure --profile my-profile-name

Or if you can and will be using it always, export an environment variable AWS_PROFILE containing the profile name like export AWS_PROFILE=<your-profile-name>.

export AWS_PROFILE=my-profile-name

And cheers, you can now use multiple credentials through AWS CLI.

Conclusion

Hi guys a quick tip from me: sometimes, having multiple projects can cause headaches. So as a reminder, always focus and finish one project first before you move to other projects. Stay tuned for more blog updates and series.