Why Quality instead of Quantity
We spent a lot of time brainstorming on how to provide the most value possible last week.
The goal is simple. Developers spend a lot of time designing a database schema, a vital component for most software projects. We aim to make that process easier.
Hence, DbSchemaLibrary was born. A huge library of database schema that developers can use to find a schema they can reference.
Original plan
The original plan was to collect a lot of schemas into a single place. As hinted in our marketing material (see below), we wrote that there will be more than 200 database schema.
Why? Well, the more there is, the more developers can reference from… right?
And so we started compiling database schemas from many different sources.
What is valuable?
When we started, there are 3 looming questions we had to answer. Answering those questions led us to learn about how developers might use DbSchemaLibrary. Now we believe that we should focus on quality instead of quantity.
These were the 3 questions:
Q1. How should we store the data?
We had 3 storage requirements that had to be fulfilled for the schemas we’ve aggregated.
- It needs to be stored in a common format.We don’t want to be maintaining different data for each database. Who knows what will happen in the future.
- We need to be able to generate as many things possible from our stored dataIf a schema is generic enough, we want to be able to show that schema in as many database dialects as possible. PostgreSQL, MySQL, etc.
- We need to preserve the restriction if presentAt the same time, many schemas will only run on a single database. For example, PostgreSQL has a lot of extensions that wouldn’t work with other databases.
As we were designing the schema to store this data, we thought: I wish we could use our product for this!
This was a great opportunity, as now we’re thinking about what we would do ourselves.
Q2. How should the schema be searched and categorized?
Our use case is straightforward. At its core, we store a list of database schemas. Are there examples out there of something similar?
- A database schema library – If there was, we wouldn’t be building this!Assuming there’s something similar, we would be looking at what kind of tables we would expect to include. For example: If we’re building an e-commerce website, we’ll want to see an existing e-commerce system and see what kind of table they have.
- Something that stores a list of thingsWe could also find based on the functionality. Is there a database of something out there? Maybe a list of books, cars or anything indexed. We call this a catalog.
The schema will be categorized based on those 2 findings. You will be able to **search for similar systems like E-commerce, CRM, Dashboards, Blogs and Analytics. You will also be able to search based on what we call modules.** More on this later.
Q3. How should the schema be presented?
Once we find the schema, what information is important? For us, there are 2 things we’d like to know depending on the circumstances.
- What tables are there, and what are their relationships?
- What specific columns do they use? Should we include those in our design?
What’s important is understanding how things work. It’s not that important for us to copy their exact schema. That’ll be convenient, sure. But it’s not exactly a big deal to write migrations for a few tables if we know what they entail.
The best way we know to show this data is through an Entity Relationship Diagram (ERD). We’re experimenting with this so let us know if you have a suggestion!
Focus on Quality
Alongside those learning, we also wondered, should we include low quality schemas? Maybe the project has been abandoned. Maybe it hasn’t been used in production. Or maybe it’s a small schema with only 1 or 2 tables. Should we still add them for completeness’ sake?
If we wanted quantity and boast the number, the answer to all that is yes. But quantity itself is not always good.
- It could be hard to weed through low-quality schemas
- There can be many duplicates that do not add value, wasting time.
As an example, the databaseanswers.org website provides around 1400 different database schemas (now offline). The problem is that most of them aren’t real applications. Who knows what flaws are present in the schema? They are also often oversimplified. But the biggest problem by far is the difficulty in navigation.
So what’s valuable? Providing high-quality schemas that are easy to search and trivial to understand.
We’ll probably still have more than 200 schemas though!
Modules
It is clear that we need to focus on search and categorization. Tags and System search will be useful. But what we want to focus on most is Modules.
Modules are small parts of an application that are reusable across different systems. For example, many applications store user data and authentication. So we have a module for that.
Another system may want to track user behavior. For that, we’ll have the analytics module. What about payment? Job queue? We’ll have a module for each of them.
The goal is to identify all common modules that a system may ever have. So when you’re building a new system, you can pick several modules which make up the whole schema.
Mini example
Say we’re building a new SaaS project. What database tables do we need? It may use a few modules:
- User authentication
- Catalog (for whatever the SaaS is about)
- API Access
- Analytics
- Subscription Payment
- And a few more…
This way, you’ll always be able to find a good reference for whatever software you are currently building. At the same time, we will also have a domain or system-specific search. The problem with that approach is that you won’t be able to find anything for novel systems/fields.
List of modules
Below is an exhaustive list of modules we intend to provide. See something that is missing? Let us know.
What’s next?
We’re compiling dozens of schemas to fill all the modules above! We’ll need to do quality check and remove unneeded ones. They’ll also need to be tagged and categorized.
We’ll release the early access in the coming week or two. Register now to get notified!