Make Subfolder A Git Submodule // First Mission

Ever been in a situation where a sub-folder of your git repository needs to branch out as a new repository? It usually starts the same way: a monorepo that was cozy at first, then grew, and now every clone, every test, and every history search drags through code you barely touch. Here in this article, I’ve tried a new way using a python module to simplify the process. This step is also recommended by the Git core team as a way to move a sub-folder to a new clean repository (link).

We’re just clones, sir. We’re meant to be expendable.

~ Sinker.

Come on, let’s jump in! 🚀

✅ Prerequisites

First of all, you must have a Git1 on your machine. Second, must have an existing test git repository and Python 32 installed.

If you don’t have Git yet, you can install Git from its official sources; it’s available on all platforms, even on Android. Or if you have Visual Studio3 installed, just locate it from your drive. Python can also be installed using the Visual Studio installer.

🛠️ So where do we start?

This will be our initial test repository structure:

+-+ root
  |
  +-+ test-repository
  | |
  | +-+ desired-directory
  |   |
  |   +-+ contents
  | +-+ other-directory

The first step you need to do is clone the test repository by either copying it with the cp command or by creating a duplicate cloned copy using git clone. The duplicate is important: we are about to rewrite history, and you do not want to do that on your only copy.

+-+ root
  |
  +-+ test-repository
  | |
  | +-+ desired-directory
  |   |
  |   +-+ contents
  | +-+ other-directory
  +-+ test-repository-copy
  | |
  | +-+ desired-directory
  |   |
  |   +-+ contents
  | +-+ other-directory

Then install this python module named git-filter-repo. Install the module using the pip utility:

pip3 install git-filter-repo

This git-filter-repo simplifies the process of filtering files, directories, and history. This tool, as said on its GitHub page, falls in the same category as git-filter-branch. You can check its GitHub repository page for the pros and cons against similar tools, but the short version is: faster, safer, and much more pleasant to use.

Next thing we do is go into the cloned test repository and filter the directory you want (in our case it’s the desired-directory) to separate it into a new repository:

cd test-repository-copy
git filter-repo --path desired-directory --subdirectory-filter desired-directory

This will modify the cloned directory’s history and delete existing content that does not match the subdirectory filter. The new structure of the directory will be like this:

+-+ root
  |
  +-+ test-repository
  | |
  | +-+ desired-directory
  |   |
  |   +-+ contents
  | +-+ other-directory
  +-+ desired-directory
  | |
  | +-+ contents

The desired-directory will now become its own repository, retaining the history of the files that are inside it. Every commit that touched those files is preserved, rewound, and reparented, so the new repository does not feel born yesterday.

After moving the sub-folder to its own repository, we go back to our original test repository and delete the filtered directory:

cd test-repository
git rm -rf desired-directory

Still on the test-repository, create a new git submodule and link the filtered directory repository:

git submodule add ../desired-directory desired-directory

That’s all the steps needed, check if everything is working. The submodule pins a specific commit of the new repository, so the monorepo and the extracted project can now evolve separately while staying connected.

📋 Quick Review

Here are the simplified steps based on the above:

Make a copy or clone the current project where the sub-folder is located.
Install git-filter-repo using the command pip3 install git-filter-repo.
On the cloned project folder, filter it based on the directory you want to make a new repository with the command git filter-repo --path <new-path> --subdirectory-filter <filtered-directory>
Go to your current project folder and delete the sub-folder using the command git rm -rf <filtered-directory>.
On the current project create the sub-module using git submodule add <new-repo-url> <filtered-directory>.
Check if everything is okay.

That’s all guys, always make a backup of your repository before proceeding. Anyways, it’s on git version control system, so you can go back and re-fix if there is something wrong.

🧠 Final Thoughts

There are many answers on the internet regarding this matter, but mostly they don’t explain what will occur when you run this command. This one I’ve personally tried, as before I was using a mono repository setup, but it became so large that it was hard to maintain, especially in cases of testing and checking the history.

The whole trick is doing the surgery on a copy first. Filter, verify, and only then swap the original. With git-filter-repo doing the heavy lifting, what used to be a ritual with filter-branch and its reams of warnings becomes a two minute operation. Let me know in the comments if you have questions or queries, you can also DM me directly.

Follow me for similar article, tips, and tricks ❤.

Git (ɡɪt) is a distributed version-control system for tracking changes in any set of files, originally designed for coordinating work among programmers cooperating on source code during software development. ↩︎
Python is an interpreted, high-level and general-purpose programming language. Python’s design philosophy emphasizes code readability with its notable use of significant whitespace. Its language constructs and object-oriented approach aim to help programmers write clear, logical code for small and large-scale projects. ↩︎
Microsoft Visual Studio is an integrated development environment (IDE) from Microsoft. It is used to develop computer programs, as well as websites, web apps, web services and mobile apps. Visual Studio uses Microsoft software development platforms such as Windows API, Windows Forms, Windows Presentation Foundation, Windows Store and Microsoft Silverlight. It can produce both native code and managed code. ↩︎