I’m currently working on a project that’s very service oriented. That is, we’re building an app through smaller components that fit together to run a larger app. This lets me write small, focused code and use different languages and tools very easily on the backend while end users see one unified app. This isn’t always the best way to do things and there are pros and cons in the monolithic vs. microservices debate but in this case it works well for us. One problem I’ve faced, however, is how to manage our front-end assets that are shared between multiple projects. There are a lot of good options for solving this problem but in the end I chose Git’s subtree feature. Today I’ll give an overview of what the other options are, why we chose subtree over them, and how to use Git subtrees to manage a shared dependency.
Options for sharing code between projects
We could host a private Bower registry but as the lead developer and only guy who knows how to do that stuff I really felt my time was better spent elsewhere. Right now the only viable options for hosting a private Bower registry involve running them on Python, Node.js, or Ruby. That’s not a big deal but it wouldn’t work out because:
- We didn’t want to add yet another server to our fleet just for this purpose. We couldn’t use an existing server because…
- Our current set of servers running Nginx were all reserved for public-facing production apps. We’re keeping our internal code off of publicly accessible servers
- The one free server we did have was already pretty resource constrained as it’s hosting our code with GitLab
- None of the existing servers we have set up were capable of running the software needed for a private Bower registry. The one candidate we had left just happened to be our only Apache server which was running Cachet which is a PHP app. I sure as hell was not going to set up and run Phusion Passenger or set up Apache’s mod_proxy. It just felt like a dirty solution and of course that’s my personal preference.
So instead I opted for a Git subtree.
Git subtree for dependency management
Git’s subtree feature should not be confused with submodules. I’ve worked with submodules before and not a single one of those projects existed for more than a few days after initializing the submodule. There are lots of great articles out there explaining why Git submodules such so I won’t get into them here. Basically, a subtree is like a repo within a repo except that it belongs to the main repo and its own repo at the same time. That sound confusing as hell, I know. Let me break that down here.
Let’s say you’ve got a project set up like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17
I set this up very similarly to the project I’m working on now. So within your project root you’ve got a folder in
public/less that holds your LESS files that end up being compiled into the project’s final stylesheet. But now you have another project, a static website, that holds the canonical version of the project’s styles and you want both the web application and static website to share the same styling but still be able to diverge from one another where necessary. For example, your web app will have certain modules like form controls and such that simply don’t exist in the static site. So you want the web app to use the same base styling as the static site but also be able to add additional styles specific to that project and remove styles it doesn’t need all while sharing the same exact
Supposing the static site is laid out like this, here’s how you do that:
1 2 3 4 5 6 7 8 9 10 11 12 13
As you can see, both sites have slightly different folder structures. We want to share the entire
less/ directory of the static site between the
static-website project and the
webapp project. We could symlink them together and call it a day but that’s lazy and creates more problems down the road. Our goal here isn’t just to share code but to manage the styles as their very own project so if you were to create, say, a new blog outside of the static site and the web app you could clone the styling project into the blog theme and and be up and running with a new theme in no time without messing around with cloning different projects just to copy over the stylesheet files. The other benefit is that you can update every project that uses these files with one simple command rather than the careful copy/paste job you’d be left with if you used a symlink or managed this manually.
Create a submodule
The first step to this ideal code sharing world is to break out the shared code into its own repository. So first off, create a new bare Git repository either on your local machine, GitHub, or on any Git-enabled server you own. I run a copy of GitLab on a server so I made on there. I’d recommend you initialize the new repo on your own server or on GitHub rather than locally.
Once you have a new remote repository available hop back on over to your project code. In our case, the static site’s
less folder is what we want to share. We’re going to replace the
less folder in the web application with the files from the static site and then I’ll show you how each of the projects can still have separate stylesheet code that’s specific to them all while sharing the base code.
Now, within the project that holds the files you want to share (for us that’s the static site), move the files outside of the repository into their own, non-Git-tracked folder. There’s a few ways to do it but no matter what you want to avoid a situation where Git suddenly can’t find your old files because you outright deleted them or moved them. To avoid this, copy the folder to its new location (something like
cp -R project-root/path/to/shared_code /path/to/folder/OUTSIDE/OF/project) and then, back in the root of the repository, you’ll simply delete the entire folder you just copied out with
git rm -r /path/to/shared_code. Make sure you commit right here so you can rollback if something goes wrong later.
In our example scenario we would now have 3 folders, two of which are under version control:
1 2 3 4 5 6 7 8 9 10 11 12 13
shared_code is not under version control. Let’s do that now. Enter the folder, initialize the repository, and add the new GitHub or other repo you created earlier as the remote.
1 2 3 4 5 6
Okay, great. Now the shared files are available remotely. You can clone them as their own project or you can add them as a subtree to existing one. We’re going to add this new repository as a subtree on an existing project.
First, enter the first project that’ll use the shared code and run these command to add a subtree (in our case we’ll add the shared code to
webapp and then
NOTE: In our example, we already have a
less folder in the
webapp project. So before we add the subtree we delete the entire folder. In your case you may have files you want to merge in. If that’s the case, copy those files to a safe place and we’ll talk about the merging of them later on.
1 2 3
Here what we’ve done is entered our main project and added a new remote repository (
git remote add -f) that’s specific to only one path in your project. That’s what the
--prefix option does. We defined where that remote lives (the
firstname.lastname@example.org:/... part) and now we can refer to it as
shared_code when we want to clone it or push/pull to or from it. If you don’t do this first step then you have to type the path to the remote project each time which is lame.
The next git command adds the subtree. Notice that we refer to it by its remote name and branch (
shared_code master). From this point on, we can work with this remote as if it were part of our main project and it will always put the changes into the folder specified in the
--prefix flag. That last flag,
--squash tells Git that we don’t want the entire history of the
shared_code project when we pull it down. After all, the main project’s history is what we’re concerned with. You can manage the shared code’s Git commits and history from the project itself.
Now all we need to do is pull down the code for that project. That’s as simple as working with any other Git repo. Just a quick…
We update the subtree’s history and then pull down the actual code making sure not compress the subtree’s history into one single commit. Now we have a bunch of stylesheets in our
less folder just like we had before only now we can pull in changes that happen from other projects any time and we can push changes to them from our main project here!
Now let’s say you’re changing the styles and you’ve made a change that needs to be incorporated into all sites that use the shared code. You’ll now want to push to the remote so other projects depending on the shared code can be updated. Unlike with submodules, this is really easy. All you have to do is…
To get your other projects depending on the same shared code you can just follow these steps in each project that needs the shared library or code. The cool thing about subtrees is that the code you’re managing as a subtree gets cloned and tracked along with your main project so you don’t run into the problems you see with submodules.
The only things you need to remember when using subtrees are:
- Never rebase the repo. If you do this then the main repo will lose track of the subtree and you’ll have to add it all over again.
- Be careful not to mix commits meant for the main project with commits meant for the subtree. Doing so will make the commit history really confusing.
Using shared stylesheets with Git subtrees
So I promised to explain how you could have two or more projects all sharing the same base CSS (LESS, actually) by using a Git subtree while retaining each project’s ability to override and ignore styles as well.
Our goals for this are:
- Every project uses the same remote Git repo as a subtree for their styling
- Any project can choose not to use some of the code without adding extra code to the other projects
- Any project can add additional CSS that’s specific to the project without forcing the other projects to use that code
One way to do this would be with Git ignores. But I prefer all the projects to have custom code available but not enforce its use. To do this, we just have to structure our LESS files properly.
So every project has a
less subtree that looks like this:
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15
In this example,
style.less simply uses
@import statements to pull in style rules from the
partials folders in the correct order in order to create one coherent CSS file when compiled. So assuming this is the base set of styles shared between each site, the only thing that a different project needs to do in order to have their own custom styles is add a new “master” stylesheet file with its own set of
The way I’ve been managing this project is to have each project add any additional rules that wouldn’t apply to the other projects in their own LESS files prefixed with the project name. So back in the shared styles project you end up the base set of styling plus any files added by separate projects. Some may argue that this makes the project unfocused and bloated. I like it because it allows me to reuse a style from a different project at any time if I end up using a new UI element from that project.
Anyway, to illustrate this example, let’s look at two example LESS files. One is the default stylesheet used by the static site and the other is main file used by our web app.
1 2 3 4 5 6 7 8 9
Pretty straightforward. Now what if the web app does not use the same footer and needs extra form controls? Easy. It just creates a new
webapp.less file and uses it instead.
1 2 3 4 5 6 7 8 9
In the code above we just replaced the
footer.less import with our special
forms file. That’ll all get compiled as CSS to whatever directory you need and you get all the benefits of a shared library with all the flexibility of separate stylesheet code.
You can use this same idea in a lot of different ways. This is only one very specific use case. Just remember that before you reach for a subtree, maybe instead you’d rather use Bower, NPM, or some other package management solution.
Note: I wrote this to help me better understand these concepts myself. I am not a Git subtree expert. I just use them and writing about them helps me understand them even better. Please let me know if you spot something wrong in this article.