Current loctn: about > devel motivation

Development Motivation

Content for this page is still being edited.

The motivation for the development of the SelCtlDB is derived from experiences of several year’s work with a wide variety of non-profit databases. This page outlines the problems we have seen fairly frequently.

Data Model Deficiencies

We have frequently encountered the following deficiencies with the data structures implemented in non-profit databases:

• Separate columns numbered 1,2, etc are often used instead of a more rigorous and flexible approach using separate rows in a related table:
  • names of individuals
  • phone numbers
  • email addresses
  • mailing addresses/location descriptions
  • notes from communications
  • languages used
  • racial background
  • ethnicity
• Information detailing multiple direct contacts with a party (phone calls, personal meetings, etc) is often stored as separate sentences in a single, large remarks field,
• Data is often not maintained in a way that does not allow retrieval of historical information. The typical methods are capable of only storing information as it exists at the current time:
  • board membership
  • committee membership
  • organization membership
  • task group membership
  • subscribers to newsletters, etc.
• Phone numbers are typically associated with a person directly even though this is typically correct only for mobile phones. Most nonprofit databases do not offer any possibility of associating standard hard-wired phones with a specific location, even in cases where the database otherwise provides for entry of multiple locations
• Phone numbers are typically stored in a single text field often including multiple numbers, abbreviations and notes on use of the contained phone numbers, and with varying uses of separators (dashes, parentheses, decimals, spaces) and inconsistent storage of area codes, country codes, extension numbers. This makes use of phone numbers in reports, data exports, etc very difficult.
• Tracking of important personal events is of high importance for organizations employing a high level of engagement with constituents, but few databases provide a formal, easily searchable approach for storing such information - the typical approach is to use free text entries in a notes field.
• Tables, fields and other database objects may be named in ways that are known to cause frequent problems:
  • using special characters (?, +, :, etc.) in names
  • using reserved keywords as field names (For instance the word date frequently has a special interpretation in database systems. Nonetheless, it is fairly common for nonprofit databases to use the word date as a field name.)

It is common for these situations to present real problems as the amount of stored data increases, or as the organization's data needs change. These problems often arise from efforts to save money and time under a small development budget. Alternatively, they may arise because a relatively inexperienced employee or volunteer developer does not understand the potential problems that may arise.

Constantly Recreating the Wheel

Nearly every non-profit database tracks a similar set of core data:

• personal information like names
• location addresses
• phone numbers
• email addresses
• records of contact with constituents
• donations
• individual contact preferences
• membership on committees, action teams, etc.
• membership on subscription lists for newsletters, etc.

Yet most non-profits start from scratch in developing data structures to cover these needs. An expensive and time-consuming process is undertaken to build a set of structures that has already been constructed many times previously.

Many of these projects then seek short-cuts or other economy measures to stay within budget. Rigorous coverage of the basics is often sacrificed in order to preserve funds for more idiosyncratic needs. However, short-cuts on the basics often manifest in predictable problems as an organization's data needs increase over time. As a result, few of the non-profit databases now being used employ a rigorous, complete data model that would save money in the long run.

If a freely shared, rigorous, comprehensive data model were available for reuse by a large number of developers, this frequent process of introducing shortcuts might be eliminated. The availability of a reuseable data model could allow a project to start with a substantial amount of the data modelling work already completed in a rigorous fashion - allowing each project to focus more funds on customizations required for truly unique organizational needs.

The amount of time this might save is substantial: over 500 hours were spent developing V1 of the SelCtlDB, and over 900 hours were spent on continuing development leading to V2. There is a potential major savings to be realized by adopting an existing, rigorously designed system such as the SelCtlDB as the basis for a development effort, then focusing funds on essential customizations.

Redesign is Harder Than Initial Construction

Redesign of an existing database is often more expensive than the initial effort to design and implement a more comphrehensive application. A substantial effort may be required to simply understand the current application and it's data contents. This is is common if:

• the database has little accompanying technical documentation
• a chain of developers has been used
• inconsistent development and data entry approaches have been used
• users have improvised ways of entering data for which the database was not originally designed
• work processes that make use of the database are unstable or poorly defined

Poor Database Conventions

Naming Conventions

Because most nonprofit database are each a unique creation, it is typical that each implemented database applies a different set of database naming conventions. It is also relatively common for naming approaches to be inconsistent within the same application. For instance, the name NameFirst may be used in one place, while the same information is represented as FirstName or fname in other locations.

Table/Relationship Structures

It is quite common for tables in non-profit databases to have poor construction. For instance, tables are often defined with key structures likely to cause problems, or with no keys defined at all.

Motivation - Filling the Data Model Gap

The data model for the SelCtlDB was developed as an attempt to fill the needs outlined above: To provide a complete, rigorous, flexible, general data model that is managed in a way that allows very clear documentation and understanding to facilitate reuse by a variety of developers working in a variety of DBMS systems.

Last modified: 14:12:25 23-Jan-2009