Kapost Engineering

Recent Posts


Recent Comments


Archives


Categories


Meta


Organizing Large React Applications

Nathanael BeisiegelNathanael Beisiegel

As you may have picked up from previous posts, we have embraced React here at Kapost. Our development teams have grown the React codebase quite a bit over that past six months. React, of course, does not dictate any folder structure but we have felt an increasing need to organize new and existing apps as we grow and share more code.

Why does application organization matter when we have editors with project search and IDE “go to definition” commands? From a human perspective, it’s easy to get lost in hundreds of files. Clear and effective organization helps us focus on just one aspect of the complex system. It can be easier to onboard new developers who can learn parts of the app in isolation. From a machine perspective, it’s simpler to manage dependencies and to optimize code with build tools like webpack. In short, it provides the usual benefits of good abstraction—comprehensibility and composability.

Organization doesn’t apply just to the source code—assets, domains, developer tools, and build artifacts also play a (sometimes major) part of the application structure. I’ll primarily focus on JavaScript organization but will also consider the application organization as a whole.

JavaScript Organization

Before I go too deep into the weeds, let’s start with this top-level project skeleton:

app/
config/
package.json
README.md
... and tons of project .dotfiles and tool files ...

Yes it looks incredibly Rails-like—let’s not worry about the top-level structure and naming for now (see footnote 1 for discussion on naming). What’s important is that all actual application code lives under app/. Application configuration will live under config/.

There are a couple primary patterns for organizing projects out in the wild. First up is I what I believe is the more common pattern: File-Type First.

File-Type First (FTF)

The File-Type First pattern (abbreviated here as FTF) is simply arranging your application files by their function. Each conceptual type of file lives under a common parent that other file-types do not. For example: all React components would live under components/, and all Redux reducers would live under reducers/.

app/
  reducers/
    root.js
    memberships.js
  components/
    home.jsx
    memberships.jsx
  ... of course more type folders/ ...

Had enough Rails yet? Discounting the specific top-level naming, many new React projects start with this structure too. The Redux examples by Dan Abramov all do, as does Erik Rasmussen’s React Redux Universal Hot Example, Rick Wong’s React Isomorphic Starterkit, and Konstantin Tarkus’ React Starter Kit.

This pattern is easy to start with when designing or learning a new framework or library—you know where to put things from the start. Even with an empty project it provides some structure. Of course, things can get hairy with scale and hundreds of files.

app/
  reducers/
    index.js (can have a index file to require the rest if logical)
    1.js
    2.js
    3.js
    4.js
    5.js
    ... hundreds more ...
  components/
    ... so so many ...

We could certainly organize with folders / namespaces.

app/
  reducers/
    index.js
    auth/
      memberships.js
      1.js
      ...
    commentable/
      comments.js
      2.js
      ...
    ... many more folders/ ...
  components/
    auth/
      login.jsx
      ...
    commentable/
      comments.jsx
      ...
  ...

Off the top of my head, there are a few apparent benefits of FTF.

  1. Easy initial organization, as I mentioned above.
  2. There is no question where any base files might be, which can be especially useful if you are using inheritance patterns.
  3. Files that import and compose many other same-type files into one main file (like Redux reducers) have nowhere else to look but the current folder.
  4. If you have any autoload behavior, it can be substantially easier to point to just one folder based on name. This is important in frameworks like Rails that may offer naming and location convention, but much less so without a major framework dictating file placement.

However, I would argue that FTF has many shortcomings, especially at scale. Here are a few.

  1. Module groupings must be repeated. You’ll always have to repeat organization in some sense, but if you want to stay consistent, you have to be disciplined and repeat whatever that module was named.
  2. You don’t have to group related features. It’s easy to start with a new feature and forget to group it—or drop it in a module with dubious grouping relation. Splitting up the app later becomes a big search game if we weren’t disciplined.
  3. Naming modules is harder than naming a feature. Do we stick with nouns? Words with a -able suffix? Any pattern at all? Are modules named for a pattern or a feature? If so, what happens when a file no longer really fits this pattern? Do I then have to move all related files out of this module across the app? Does my team know the naming scheme I keep in my head?
  4. Drawing application lines for optimization is a lot more difficult. Possibly the biggest concern—we will likely need to break up our app as it gets bigger. We may need to add multiple JavaScript entry points or code-split our front-end app through routing and webpack’s require.ensure. When you have all files grouped by file type, there are no clear application lines to split on from project structure.
  5. Boot files and entry points are not obvious. What type of file is the entry point? Is it named boot? I have to know this up front or do some searching in my app configuration.
  6. Editing a feature means editing files in many top-level folders. Minor pain of course, as “Project Fuzzy Finder” is a ⌘T away. Let’s hope that naming was consistent or that our entire team prefers heavier IDEs with “go to definition” features.

Perhaps some of these criticisms are unfair to pin directly on FTF. And, you might say, we could add application-level grouping to our big app. Let’s look at another pattern and then address these points.

Feature First (Pods)

If have looked at Ember at all then you may be familiar with the concept of pods, where the feature comes first. Our top level folder is the feature name and each pod may be organized with any file types the feature may demand.

app/
  authentication/
    api/
    components/
    helpers/
    ...
  comments/
    actions/
    api/
    components/
    reducers/
    stores/
    ...
  ...

Anything related to comments is conveniently located right within the comments/ folder. As soon as I open a folder I get an immediate idea of how that feature is implemented and what files to edit for that feature. Taking that a step further, I like to also include the primary entry point(s) of a pod at its root, if applicable. Usually this is just a container component. This makes it easier to follow how a feature works and how to (re)use it.

app/
  app.jsx
  authentication/
    authenticationContainer.jsx
    actions/
    ...
  comments/
    commentsContainer.jsx
    actions/
    ...
  ...

Grouping is fine too—just features of features.

app/
  settings/
    profile/
    notifications/
  ...

Pods solve many of the issues with FTF, in addition to some other benefits:

  1. Clarity of feature naming. It’s still up to you to name features, which can be hard. But you just do it once, and you don’t have to remember folder names to maintain grouping across the app.
  2. Framework structure independence. Have a new technology or pattern? You don’t have to have a bunch of top level folders for every file-type imaginable. You can simply organize each pod to use the frameworks and abstractions you have chosen for the feature. It’s a lot less confusing than FTF, where you would have different features inconsistently using different top-level folders.
  3. Entry file(s) are obvious at feature and application level.
  4. It’s trivial to split code in your app. It can be a lot easier to load split your code when you know what features a section of your app demands.

Of course, there are a couple pain points here too.

  1. File types that roll up have to dig all over the app. Redux, for example, needs to roll up at least one file to generate a single store through reducers (see footnote 2). I actually think this pain is a good thing, as it requires you to think about how you are sharing state across features, particularly if you have multiple app sections or use code-splitting in your project.
  2. Autoload conventions break down. Not a big issue as React apps typically don’t have this, and this would make code-splitting more difficult anyway.

Another question: where do we put base files? A somewhat inelegant solution would be to put them in a framework specific folder:

app/
  flux/
    apiUtils.js
    baseActions.js
    baseStore.js
    connectToStores.js

You could also create a separate app/ folder if you have several folders of these top-level base files. After all, it is an application concern—you can think of an app as a top-level pod.

As we’ve built out parts of our Studio app we have started to notice many of our pods map closely to our database schema / domain as well. It’s also starting to match how our back-end’s service layer is organized. It’s quite interesting to see how closely the back-end mirrors the front-end as we grow and enforce application boundaries (Rails engines split by major entities look a lot like pods to me). Of course this mirroring is not one-to-one and we don’t need it to be, but it’s nice to have code that increasingly matches the product domain.

Apps of Apps

Do pods hold up if we apply even more product organization? As you grow, you will probably want to split big sections of your app into smaller apps. For example, each top-level tab on Twitter is easily its own app.

App sections of the Twitter App

You probably haven’t hot-loaded any ‘Moments’ JavaScript.

We have multiple apps in Kapost as well.

The full suite of the Kapost application.

The Kapost full-suite.

While our currently applications are split by teams, repositories, and various frameworks, I hope to transition towards a single JS application containing all of the Kapost front-end. It would lead to an efficient Single Page Application, through code-splitting and sharing duplicate code. What might that look like?

app/
  kapost.jsx
  studio/
    studioEntry.jsx
    content/
    ...
  gallery/
    galleryEntry.jsx
    collections/
    ...
  insights/
    insightsEntry.jsx
    content-scoring/
    ...
  members/
    membersEntry.jsx
    profile/
    ...

We likely will have a lot of shared code too, that can form common code bundles.

app/
  ...
  shared/
    users/
    ui/
      dropdowns/
      ...
    ...

So far the pod pattern is still holding up. We could use the FTF pattern per app but would have the same drawbacks, just at an application level.

What about files that roll up like routes or reducers? There are elegant solutions to handle code-splitting such as dynamic routing with React Router and dynamically adding Redux reducers, but where do we put those rolled-up files? Just like with the base files, I think a top-level folder named boot/, a brand name like kapost/, or something similar is appropriate, as app/ is just another pod.

app/
  kapost/
    routes.jsx (holds and rolls up all other app routes dynamically)
    reducer.js (holds all reducers dynamically)
  studio/
    studioEntry.jsx
    app/
      routes.jsx (rolls up all application routes)
      reducers.jsx (rolls up all studio reducers across all the feature folders)
    ...
  ...

That looks great to me! This of course is a very shallow look at using Flux / Redux and React Router and there may be other challenges with code-splitting, but I believe the organization still holds.

Another possibility is splitting code by repository rather than many subfolders. We rely on having separate repositories for different apps for team ownership and measuring engineering performance. We can employ private NPM packages to wrap various client applications and import them in to our main app, routing like before. Separate packages present other challenges such as keeping dependencies in sync, but they allow you to pull out an app section into its own repository. You also have the nice benefit of knowing exact package versions and ensuring it all works together (with specs) from the primary app.

Note that if you split out apps into packages, you’ll also have to make packages for shared code. I think this is a good idea at a larger scale (we are working on a shared UI repository currently), but don’t abstract until you need to.

Application Organization

Unfortunately our environments and tools often dictate some folder structure as well. I could probably write a blog post about most of the following topics individually but will limit myself here to just organizational concerns.

API

I am writing with the assumption that these apps are totally separate JavaScript clients. If your product is at any substantial scale, your API should be in a separate repository to provide independence from consumer application concerns.

Universal Rendering

This is pretty difficult to setup, but less so at the organizational level. You will likely have two entry points at the app level: something like client.jsx and server.jsx. Also, you may have a set of server-only files that you’ll probably want to hide away in another folder. You’ll have to think about when to build and import things, but that’s primarily webpack and import configuration. One small configuration that you should absolutely setup: relative paths in Node. Setting the NODE_PATH ENV var works well with webpack. Alternatively, you could also set up webpack to also build your node files so you can use the same loaders. There are many techniques and configurations to support universal rendering (and many articles on the subject); I hope this setup gets easier in future.

Domains and Authentication

You’ll need to manage authentication across many apps, and possibly on the server and client. JSON Web Tokens or Secure cookies over HTTPS (under one primary domain name) are the most likely solutions. Again, this is mostly config, but you’ll likely want some auth code in the shared/ folder.

App Configuration

I initially put application configuration at the root and in the config/ folder. The placement is up to you, but you’ll need to figure out how to export configuration files to the parent application (probably as part of app index.js that you define as the root in package.json).

Assets

A huge pain to configure with multiple apps, build steps, development environments, and production content distribution networks. You’ll have to translate paths for each app with both development and production environments, potentially after a build step. Your build folder or static file server will also need to look within multiple asset folders across apps. This is further complicated if you attempt to auto-inline small assets with various webpack loaders. Even more complicated still is adding a development-only server for Javascript, assets compilation, and live-reload such as the webpack dev server.

My advice—start simple, and configure this stuff after you figure out the rest of your application structure, as it gets complicated quickly. Or just grab some empty project configuration from a good universal boilerplate. You will have to edit for your own application structure, but take advantage of these examples.

Styles

The exact organization for stylesheets will look a lot different depending if you choose to inline styles through React and webpack or not. I may be old-fashioned on this one, but still prefer using vanilla SASS for imports and using build tools like gulp for things like autoprefixer. I know webpack for stylesheets and npm as a build system is what’s in vogue, but I’m not sold on inlining css or having console output collide without gulp streams. The trade-offs (such as difficult media queries, requiring universal rendering for fallback, lack of cascade, difficulty for designers with some CSS experience, etc) are too great in my opinion.

Regardless of my opinion, here’s how styles will affect project structure in either case:

Without webpack and inlining

SASS or other preprocessor imports can allow for a roll-up very similar to application JavaScript. Each app repository should have its own stylesheets folder like so:

studio/
  app/
  config/
  stylesheets/
  spec/
  package.json
  ...

I am fussy about making sure component stylesheets map one-to-one with my application structure. Each stylesheet is name-spaced with a special, prefixed class to go along with the component, as it prevents a lot of style collisions.

studio/
  app/
    comments/
      commentEntry.jsx
  stylesheets/
    comments/
      _commentEntry.scss
// _commentEntry.scss
.studio-comment-entry-component {
  // my name-spaced styles
}


// commentEntry.jsx#render
render() {
  <div className="studio-comment-entry-component">...</div>
}

You can obviously have shared styles too, mapping to the shared/ JavaScript folder. Global styles should go somewhere like stylesheets/app/ and be used extremely sparingly (and preferably name-spaced like components are).

Each app is responsible for importing all of its feature styles, and the top level app is responsible for importing all sub-app stylesheets. If you split some apps into separate repos you will likely want to maintain the same build config (and sticking with one preprocessor) or you’ll have to figure out how to stitch together multiple build outputs.

With webpack and inlining

Styles really are just in JavaScript-land at this point, so you can add appropriate structure to each pod and require in the style files with a loader. Something like this following:

studio/
  app/
    comments/
      styles/
        individualComponentStylesheet.scss
      ...

Less work on the configuration side of things, but again, there are trade-offs with inline styling. You could follow this organization without without inlining as well, where SASS files live inside pods but still use SASS imports. That approach may be the best of both worlds, but global SASS like variables and mixins would have to be treated as hard dependencies that a parent file would have to import first.

Testing

There are two primary options to organize your tests. You could have a separate top-level spec/ folder that matches your application’s folder structure when creating specs. Most test loaders use a regex to match a Spec suffix, so the structure is just to maintain consistency / sanity. However, manually matching folder structure should be a red flag at this point if we are committing to pods. Why not move tests to the pod too?

studio/
  app/
    comments/
      components/
      commentsContainer.jsx
      specs/
        components/ (unit tests of sorts)
        integration/ (testing entire comment feature)
        commentsContainerSpec.jsx (container could even be the main integration test)
        ...
      ...

Your spec loader can still use the same spec regex; you just have to point to your app/ folder rather than spec/. Each app / pod that composes other pods can also have tests at that appropriate level too (integration specs are a composition of many features, just like apps and containers). Note that with this pattern you could also have each test in the same directory as the actual file that it’s testing, but in my opinion that looks a bit cluttered and only makes sense if you just write unit tests. As long as you are consistent, do what feels the best to you and your team.

Wrapping Up

Hopefully I’ve impressed upon you the many application and developer concerns that are affected by your project organization. I’m sure I have missed many things, and would love to hear more ideas or approaches to organize a React project. Please feel free to reach out by commenting or by tweeting me. Thanks for reading!

Footnotes

[1] Folder Naming

I actually don’t care about folder naming that much. I have used my naming preferences throughout the article. There was a recent discussion on Twitter for naming folders within library repositories.

Really happy to see that “modules” is catching on as a modern alternative to src/lib in popular #JavaScript repos. Makes a lot more sense!

— Michael Jackson (@mjackson) November 25, 2015

Regardless of your preferences, make sure you are consistent! You should document the pattern for your team / contributors.

[2] Ducks (Redux Reducer Bundles)

If you keep up with the Redux community, you are probably aware of Ducks. This is an effort by Erik Rasmussen to reduce the pain of having action types, actions, and reducers split across files. The solution is modules, a file that will combine and export all action/reducer pairs with consistent naming.

This is still file-type organization, and I think it would work nicely within a pod, rather than at the top-level as with FTF.

Nathanael is a full-stack developer on the Studio Engineering Team at Kapost. He is passionate about improving front-end development with better React and UX patterns. Follow him on Twitter @NBeisiegel.

Comments 9
  • Chris
    Posted on

    Chris Chris

    Reply Author

    Thanks for writing this! I’ve been looking for naming conventions for these type of project structures. We use the ‘Feature First’ approach and just refer to them as modules. One thing that is different and is nice is that we include all code related to the feature in the module, which includes CSS files, and any server side code as well – adding directories as necessary. It really makes browsing the directory structure easy.


    • Nathanael Beisiegel
      Posted on

      Nathanael Beisiegel Nathanael Beisiegel

      Reply Author

      Hey Chris! Thanks for sharing. We started a new project after this post was written, and our css pattern evolved to following:


      app/
      app.scss // entry, pulls in deps and rolls up all pod style/index.scss files
      authentication/
      styles/
      _index.scss // roll up styles within dir
      _otherStyles.scss

      I really like this pattern; we use the index pattern for other rollup such as reducers too. You could also put the entries at the pod root, depending on your taste (it was a little too cluttered for use and we wanted to highlight the Container entry files).


  • Jose
    Posted on

    Jose Jose

    Reply Author

    Hi, any example on github?


    • Nathanael Beisiegel
      Posted on

      Nathanael Beisiegel Nathanael Beisiegel

      Reply Author

      Hi Jose!

      Unfortunately not yet! We hope to release our own boilerplate within a month or two (with some more tutorials to match). Check back soon!


  • David Plunkett
    Posted on

    David Plunkett David Plunkett

    Reply Author

    This is great, thanks for sharing. Can you recommend any decent (simple) boilerplates that use the Pod structure successfully, I have been searching and struggling to fine any. It’s strange as this folder structure has become increasingly popular within the Angular community over the last few years and I was surprised to see so few people using it for React, keeping related files in one place just makes so much sense. I agree about still using Gulp to compile the Sass, inlining the css within components definitely seems the trendy thing to do, but you need to jump though a ton of hoops for things like mixins and media queries which has put me off for the time being.

    BTW The link to the Dynamic Routing is coming back with a 404


    • Nathanael Beisiegel
      Posted on

      Nathanael Beisiegel Nathanael Beisiegel

      Reply Author

      Hi David!

      Unfortunately I haven’t seen any boilerplates that follow this structure in a way I’m happy with. We are building a new project following this pattern here at Kapost and hope to release our own boilerplate within a month or two (with more posts and tutorials on usage). Check back soon!

      Thanks for heads up on the 404, it should be fixed.