Archive for Data Quality Management

How to Develop a Data Governance Policy

Do you have a data governance policy for your association? In this post, we’ll discuss why data governance is important and what your policy should include.

Why You Need a Data Governance Policy

Data is an asset, just like cash, buildings, and people. Just like other assets, data requires strong, consistent management.
When we neglect data, the results aren’t pretty – you get data quality issues, conflicting information, and confused staff who don’t know how to get answers. The result is lower quality data that causes mistrust and frustration. Staff turn to other means — intuition, politics, and tradition — when it’s not easy to make data-guided decisions and that can cost your organization.
Data governance is a cross-functional management activity that, at its core, recognizes data as an enterprise asset. A data governance policy will ensure that your association is treating data as an asset.

Developing a Data Governance Policy

Data governance policies can be authored by an internal team. Although, you may want to consider hiring an outside consultant if you have a large amount of data and data systems and/or would like a third party partner to provide an objective perspective. Here’s a 10-step process to developing your own policy.

  1. Communicate the value of data governance internally to business users and leadership. If your organization doesn’t currently have data governance, you may need to establish a business case. Consider the cost of the current situation and also the possible savings if your organization had data governance.
  2. Build a Data Governance Team. An internal team can help manage data governance and help ensure cross-departmental support.
  3. Assess the current state of data governance in within IT departments and business operations.
  4. Determine roles and responsibilities. A RACI chart could help you map out who is responsible, who is accountable, who needs to be consulted, and who should be kept informed about changes.
  5. Establish expectations, wants, and needs of key stakeholders through interviews, meetings, and informal conversations. This serves two purposes – you get valuable input but it’s also an opportunity to secure buy-in.
  6. Draft your policy and ask key stakeholders to review it and endorse it.
  7. Communicate the policy to all stakeholders. This could be a combination of group meetings and training, one-on-one conversations, recorded training videos, and written communication. Remember to consider other’s learning and communication preferences when selecting how to communicate.
  8. Establish performance metrics and a way to monitor adherence to the policy.
  9. Review performance regularly with your data governance team.
  10. Keep the policy fresh. Regularly review your data governance policy to make sure it reflects the current needs of the organization and stakeholders.

What Your Data Governance Policy Should Include

Unsure of what to include in your written Data Governance Policy? There are a number of factors to consider when developing a data governance policy and ensuring that data governance is adopted at a cultural level. Here’s an outline of a policy that you can adapt for your organization.

  • Goals – Establish overarching goals for each area below. Establish performance metrics so you can evaluate success.
  • People – Define the key data-related roles throughout the organization. For every data system, identify the data stewards who manage data as a corporate asset and focus on data quality; the data owners who have decision-making authority and define data quality standards; and the IT staff who provide technical support and monitor compliance. Consider using a RACI chart for this section.
  • Data Inventory – Inventory and document all data sources. Regularly review the inventory to include new sources and remove old sources.
  • Data Content Management – Identify purposes for which data are collected. Communicate purposes to staff and customers. Review collection policies regularly.
  • Data Records Management – Develop and adhere to policies that define how records should be created, maintained, and deleted.
  • Data Quality – Assign responsibility for data quality to appropriate staff. A data steward should perform regular audits to ensure quality.
  • Data Access – Define permissions and who has access to what systems.
  • Data Security – Define policies around data security, sharing of data, access to data. Include a risk assessment in this section that indicates the risk and the probability of risk occurrence.

Graphic Designed by Freepik

Why Your Data Needs a Clean Bill of Health

You’ve found the data you want to analyze and have brought it into a data mart. You know how to use visualization software to turn the data into informative pictures. You know the kind of things you want to find out from your data.
Even so, you’re not quite ready to go forward with your analysis.
Before you get started breaking your data down, you need to understand what you have and get it into good shape. Otherwise, you won’t be analyzing anything. You’ll just be sitting in front of your computer, frustrated that your visualizations aren’t telling much of a story.
Cleaning
This is called data cleansing. And though it may seem like an obstacle on the road to turning your data into information, you won’t be able to perform an analysis without doing it.
Here are some steps to follow in order to make sure your data is ready to go.

  • Figure out the fields in your data. What is contained in each row and are there any problems with it? Each entry should have some sort of unique key associated with it. Is the key duplicated anywhere? Are there many fields that were left empty? And is there even any data there? Duplicated keys and items with too many nulls can cause headaches in your analysis. Likewise, a database that may have been started years ago but never received updates probably isn’t worth the effort to try to analyze. You can help prevent problems of unexpected missing values by adding terms like “unknown” or “not applicable” in their place. This way, you won’t miss out on counting a record because data is missing, and you’ll have more context to analyze what is there.
  • How is your data formatted? This is one place where you need to be a stickler for detail. Are all dates written the same, or does your data source interchangeably use “March 21, 2016” and “3-21-16”? Are addresses and zip codes all done the same? Are there any problems with values – like birth dates of “1900-01-01,” quantities of “999999,” and zip codes of “00000”? If items don’t make sense or aren’t consistently entered in a way that your visualization software can recognize, the data is worthless for your analysis. You won’t be able to make use of date or geographical visualization functions, and your results will be thrown off by obviously incorrect outliers.
  • Can you validate the data? While you may work hard to get the data into a format where you can build better visualizations and tell a better story, you should first make sure that the data you’re getting is correct. If you’re analyzing membership data, it makes sense to talk to someone from that department who should know the basics of what the data says. Were there really 325 new members added in January? If the raw data doesn’t make sense to the people who are most familiar with it, you should go back and correct the problem before moving forward to more analysis. Hidden issues like counting the number of records instead of distinct values, reporting that is based on outdated data snapshots and changes in historical business processes can make reconciling the data mart information challenging. Also, processes to change data may not universally be reflected in the data. Scripts written to change individual member status values might not align with the data ranges of orders and subscriptions.
  • Is data consistently and correctly updated? Even if you do everything to fix your data in the data mart, technical processes are not always perfect. You need to monitor the data that comes into the data mart for consistency and correctness. You should maintain log tables that let you know when the data has updated and show you any problems that were encountered. You should also make use of visualization and analysis techniques to monitor ongoing data quality by tracking potential issues, like incorrect record amounts and missing values. It is important to measure accuracy on a regular basis, making sure that all changes are coming through.

Let’s Spend Some Data Quality Time Together

Gartner estimates that poor data quality costs an average organization $13.5 million per year.  Drivers of these costs include lack of a common language for business information, independent data maintenance, and multiple versions of the truth.
Associations face similar challenges as they work with data from multiple specialized systems, often including an AMS, event registration, survey management, social media, and email marketing applications.  Customer data such as demographics, job function, career stage, and company name is vital to associations as it drives pricing strategies, product offerings, and ultimately customer value.
To illustrate the impact of data quality, let’s suppose that customer records contain 50 attributes and each is inaccurate with probability of 5%.
(1-.05)50 = .078
In this example, the result is that fewer than 10% complete and accurate customer records!
Technology solutions that combine data directly from source systems for analytics are particularly susceptible to the high cost of poor data quality.  Complete and accurate data is also needed to leverage many predictive analytics and data mining techniques to ensure accurate data-guided decisions.
Like other business initiatives, successful data quality requires an optimal combination of people, process, and technology to serve as a foundation for successful association analytics.

Process

The activities which occur during the “collect” step of DSK’s methodology address data quality.  The data source inventory identifies the location and specifics of the data which is used to answer business questions in addition to reference data, such as standard job title and demographic values.  The master data management policy includes rules for acceptable value ranges, confidence thresholds for automatic linking, procedures for adding allowable values, and monitoring strategies.  The dictionary of common business terms maps to this information and further communicates a shared understanding of data.
The initial process is iterative as cleaning and deduplication techniques are applied to historical data while confirming allowable values and thresholds.  Since the best way to minimize data quality issues is preventing them at the source, close collaboration with source system groups establishes data entry standards as part of data governance.  Ongoing processes incorporate these results to immediately improve data analytics, improve efficiency, and ensure accurate capture of data history for slowly-changing dimensions.
Data quality activities also provide immediate benefits through improved operational efficiency by reducing time-consuming and tedious tasks of correcting data and identifying duplicate records.
Technology
Often important association data consists of free-form text, such as company names and job titles, which are created and maintained by customers, sometimes in multiple systems.  This data cannot simply be cleaned using basic data queries that rely on exactly matching discrete values.
Leading Extract, Transform, and Load (ETL) tools such as Microsoft SQL Server Integration Services (SSIS) provide important data quality tools that leverage text matching algorithms to clean data.
Data quality features include:

  • Data Quality Services: Knowledge-driven approach providing interactive and automated techniques to manage data quality using domains.
  • Fuzzy Lookup: Comparison of values against reference data to create similarity and confidence measures to automatically link data or flag for manual review.
  • Fuzzy Grouping: Creation of groups of candidate duplicate records assigned probability scores.

An added benefit of these tools is that data quality improves at a greater rate over time as the knowledge base of domain data and matched values grows over time, essentially creating a self-learning system.  The technology also maintains audit data to allow data quality to serve as a business area and leverage the same data analytics and Tableau visualization tools as other association business areas.

People

Incorporating data quality as a priority within the data analytics process enhances trust in data, demonstrates tangible benefits, improves efficiency, and strengthens adoption of data analytics. Like data-guided decisions, the core of these benefits is people.
DSK understands the cost of not focusing on data quality.  That’s why we include data cleaning as a core part of our proven process to position your association for success with data analytics.

How to Perform a Data Inventory for Associations

Sometimes when our clients begin a business intelligence initiative, they become more concerned about collecting new data to analyze, and focus less on the data that they already have. One reason this happens is that they don’t always know what data they do have. We often say that data is one of your association’s most valuable assets, and you want to treat it that way.  Just like with cash, you need to monitor and manage it.  Performing a data inventory is the first step to understanding your data and the state it is in. Because business intelligence is all about using data to make decisions and decisions are basically answers to question, it follows that a best practice is to determine what data you have available to answer your business questions.
groceries
Imagine you are a grocery store clerk and each night you need to perform an inventory of your store so that you can assess its current state. You start by walking around the store to see what aisles are either stocked or missing items. For your database, you will “walk around” to see what tables either have data or are empty. If that table does not have data, you will not be able to analyze anything around that topic.
Next in your grocery store, you start looking a bit closer at each aisle. When you are looking at cereal, how many boxes do you have in the aisle? For your database, you will look at your tables to see how many rows exist in those tables.   Maybe a table has just a few rows in it. For example, if you have 50K individual records in your database, but only 500 organization records, you may not be able to do meaningful analysis of organization data.
After looking at the overall cereal section, you then dive into the specifics. What is the breakdown of cereal on the aisle? Company, brands and type? Similarly, for your table inventory, you will want to look at the specific columns in the table to see:

  • Column Name
  • Data type
  • % of columns populated

For your 50K individual records, if only 10% of those records have demographic information such as gender, ethnicity and age, you will not be able to perform a meaningful analysis of the effect of demographics on an individual’s purchasing behavior. If this is an area that you know you want to analyze, you’ll know early in the process that this is an area to address by supplementing that data.
Now we want to look at the specifics of the column data, both quantitative and qualitative. Qualitative analysis can show you trends in string data. For our cereal, we look to see how many we have of each brand. 200 from General Mills, 175 of Kellogg. In our data we need to look at the top qualitative values in our columns. If you were looking at individuals, you would look at the top 10 companies represented by those individuals in your system.
We also want to look at the quantitative data. Quantitative analysis can show you discrepancies in your data. For cereal, we could look at price. What is the most expensive price for cereal? What is the cheapest cereal? Maybe you have cereal priced for -$2.75 or $100, those discrepancies are errors that become readily apparent using this type of analysis. For your data, you could look at total number of employees for your member firms. You should analyze:

  • Maximum value
  • Minimum value
  • Mean value
  • Standard deviation

Performing this type of data inventory provides a solid foundation for your business intelligence initiative. Doing this initial work will help you understand what data you have and by extension what questions you will be able to answer. And most importantly, you will be able to find out what you don’t know. And knowing is half the battle.

The Importance of Naming Conventions for Association Analytics

When designing a data mart it’s a good idea to follow a naming convention. Although this requires some thought in the design stage, it saves significant time when maintaining and enhancing the data mart with new data for analysis. Your current and future users will benefit and be in a position to use the data to deliver superior services to members and other stakeholders. Although there are a few system limitations to dictate the naming conventions you choose, careful consideration should be given to the naming due to the very important human factor. Here are a three tips to get started:Naming data marts dsk

Consider the Audience

There are typically 3 groups who will be using the data mart:
The business end users. These are the people that will never see the data mart, but will instead see the beautiful and accurate data visualizations based on the well-managed information in the data mart.
The power users. These are the people who work with the visual interface. Many of our clients use Tableau to create data visualizations and use a data mart as the data source. To be successful, they should have an understanding of the structure of the data mart and how to translate the data to provide the information needed for the business end users.
The technical team. These are the technical experts that are responsible for building and maintaining the data mart and ensuring its quality.
Each of these groups of should be considered when deciding on what to name fields, but it’s likely the power users will have the most knowledge about both the technical and the business needs and their opinion and buy in are vitally important.

Avoid ambiguity

A field called Order is not clear. Instill clarity by using names that identify if it is the Order ID, the Order Date, etc. If a field has a Yes or No value, it might be helpful to add _YN to suffix the column name. For example, Member YN clearly indicates that this is a Yes/No field and isn’t something like the member’s name.

Documentation

Data dictionary. A data dictionary is a collection of all the metadata in your data mart and is helpful for the power users and technical team. This dictionary contains an entry for all of your data elements. Some fields you might want to include: the definition, how the field is derived (if it is calculated or is straight from the source database), the name of the data source from which it came, business rules, update frequency, the data type and when it was last updated.
Common Language Dictionary (CLD). A common language dictionary is essential to define in common, business language what each field means to your association.  For example, we have seen many different definitions of a “member” – depending on whether you count them if they are in the grace period, whether they have paid or just completed an application, etc.  A list of common terms to define for associations can be found on one of our previous blogs How to Create a Common Language Dictionary
You will encounter many opinions on naming conventions and on ways to document and communicate the information about the data mart. The time you invest in the design stage can be the difference between success and failure of future projects. It is critical  to think carefully and deliberately about how you are going to name the elements your data mart to benefit your association now and into the future.

Data Dilemma: How to Avoid the OK Corral of Data Policy at your Association

There is a long-running standoff between business users and IT over who owns the data.  Does the data belong to a central department who controls, enters and accesses data?  Or does data belong to everyone at your association and anyone can enter data?  This argument will move to the forefront as business intelligence (BI) comes to your association. data dilemma dsk
Business intelligence is all about letting business users within an association have access to data they could not see before and discovering new things about how their department runs and what they can do to improve it.  So naturally with the advent of BI, an association will move towards a more open data access policy.  For those folks in IT, the thought of everyone in their association having access to data can be quite scary.  However, have you ever seen two different people try to retrieve the same information from the database and come up with two separate answers?  This is a common problem, but the cure is not to restrict who can access the data! There are several ways IT can reduce the risk of this happening:

  1. Create data views for the business user to access. IT can built the logic and ensure that all the data is being pulled correctly.
  2. Training, training, training and documentation, documentation, documentation.  Create a data dictionary with descriptions of all the fields the user will be able to see in the data view so they will understand the data they are accessing.  Create a common language dictionary that describes the meaning of common terms, such as “new member” and “retention”.
  3. Sit down with the association business user, walk them through the data and show how to use it in visualization.

As your staff starts creating visualizations based on the data that already exists, inevitably they will start to notice gaps in the data and the discussion will begin about what new data should be included.  As anyone who has worked in the IT field for even a short period of time knows, if you don’t plan how you will store and enter data, you will end up with a big mess.  There are a few things you will need to determine once this conversation starts.

  • Do you really need this data?  Will this data improve your analysis?
  • Who else will need this data?  If there are other people who may need this data, make sure to include them in this conversation.
  • Can you get access to this data?
  • How much space will you need to store this data?
  • How will this data integrate with your other data sources?
  • How are you going to use this data?

If your association is already living in the data version of the Wild West, I find one of the best ways to reign folks back in is to create dashboards based on the existing data. You can then walk the users through the limitations of the visualizations due to bad data and discuss what things they could discover if the data was in a cleaner state.  If you can provide a concrete, positive outcome that will result from following data entry rules, you can give them the impetus to change their behavior.
Starting a business intelligence initiative can open up a whole new frontier for your association.  Take advantage of this opportunity to revise and enforce your data quality management plan so your data cowboys disarm!
 

Shed Light on Your Association’s Dark Data

Dark data?  That doesn’t sound good.  How can you determine if your association has any?  Start with Gartner’s definition of dark data, “the information assets organizations collect, process and store during regular business activities, but generally fail to use for other purposes.”  Your association undoubtedly has plenty, so how do you find the dark data with the most value and shed some light on it?
The diagram below from HP shows along the X axis all the places you can start looking for the dark data.  You might also find it helpful to refer back to one of our prior posts on locating data, How to Create a Data Inventory for Associations and Nonprofits, which contains the steps to identify your data sources and determine if they have value.  If you’ve already done this exercise, then it might be worth revisiting your list.  Let’s look at some examples.
dark data dsk solutions

Web Traffic Data

Nearly everyone is collecting the data resulting from visits to your association’s web site.  If you use Google Analytics (as most do) then you have a wealth of information at your disposal including your most popular content, browser version used, screen size used, and even custom data [note our previous blog post Engage your Members with Custom Dimensions in Google Analytics] if you’ve added it to your script.  If no one from your organization is logging into your Google Analytics account and reviewing the collected data, then it is dark!  Google Analytics even provides an API so you can get your data out, load it into your data mart, and even blend it with other data sources.

Internal Support

Many associations maintain some type of ticket or helpdesk system to aid co-workers in resolving issues with the technology in place or even with broken keyboards.  If you use such a system, do you do track more than open vs closed tickets?  By mining the data here you could learn if your department has enough volume to warrant another employee or if you see the same question over and over, or if there are only 5 of your 100 co-workers making any requests.  All of those trends could lead you to taking action that reduce effort by creating a FAQ for commonly asked questions or providing more internal education/training.
Once you have located your association’s dark data, give some thought as to whether or not it contains value.  If the data can be visualized in a way that it can tell a meaningful story about how to be more competitive, how to retain more members, how to engage more members or how to elevate your association’s image, then get the flashlight or the spotlight and start shining.

Capturing History in your Association’s Data Warehouse

Now that your association realizes the importance of using data to make decisions, and has a firm grasp on your initial set of business questions, it’s time to plan how your data mart will be set up to support those initiatives. The data mart will likely be designed by your IT team or an experienced vendor, but it is still helpful for business users to understand the underlying concepts. Your technical resource is essentially constructing a safe place for a large asset to reside, so you’ll want to be involved at a high level.

Change Happens

It’s likely that some of your business questions are something along the lines of “How do my membership numbers for this year compare to last year?” or “Are more people coming to events this year than 5 years ago?” These kinds of questions require us to “go back in time” to find the answer. When we are interested in historical analysis, it is crucial to develop a method of capturing changes within the data model. Many AMS systems do a good job of capturing historical information within logs or other tables, but there may be a gap in what is captured historically and what is important to your association.

Change Types

There are essentially 3 types of changes that can happen in your system:

  1. Unchanging Dimension
    • No changes are likely. For example, Gender.
  2. Slowly Changing Dimension
    • This is the most common
    • These are attributes that can change slowly with respect to time. An example of this is price – while the price of an item in your inventory might change periodically, it’s unlikely that you change it on a daily basis. Another example is member status which may only change 1 or 2 times per year.
  3. Rapidly Changing Dimensions
    •  These are attributes that change often with respect to time. An example may be the high bid amount on an auction item during your foundation’s annual fundraiser.

Dimensional Modeling Approach

Since slowly changing dimensions are the most common, let’s explore the different methods of storing these changes.

  • Type 1 – We don’t wish to store any history. In this type, we know that the dimensions are slowly changing, however, we are not interested in storing those changes. We are only interested in storing the current or latest value. Each time it changes, we will update the old value with the new ones. An example of this may be email address which may not be needed for analysis, but rather for reference.
  • Type 2 – In this case we want to store the history of changes for the purpose of analysis. We are interested in the full history and can also extract the history of changes when necessary. In the dimension table, we add 3 new metadata columns to capture the date range the value is good for and a column to indicate if this is the latest information available. These help us determine if a particular record in the table is the latest or not, and what time period during which the record was the latest. If there is a change to the dimension, a new row is added and the old row is updated. Using this design and looking at the example below, it’s possible to go back to any date in history and figure out what the member type of each individual was at that point in time.

data mart 1

  • Type 3 – This is used to store partial history. Instead of inserting new rows into a table like we do for Type 2, in Type 3, we add a new column to the table to store the history. This can become quite cumbersome after many changes, because that will result in multiple columns being added, and the lack of date range makes it more difficult to pin point exactly when the change occurred. There is a modified version of this type where we just store the current value and the previous value. The advantage of this type is that multiple rows are not created which eases out performance concerns.

data mart2.jpg

Developing a data mart for your requirements can be an investment of time and resources. However, the rewards of well managed data will vastly improve operations, decision making, and your ability to deliver services to your members. DSK believes that data is a critical asset and we’ve devoted multiple blog postings to Data Quality Management for associations.

Getting Started with Data Governance for Associations

Data governance is a cross-functional management activity that at its core recognizes data as an enterprise asset which is used to achieve strategic and operational goals. Data governance is also one of the most talked about, yet elusive, elements in the data management space.  We know that governance is mission-critical in achieving an association’s desired data management goals.  However, initializing and maintaining a successful data governance program can be challenging for many associations.  Having a data governance program will position your association for success. data governance for associations
Understanding that your data is an asset unique to your association is key to communicating and getting buy-in for its importance throughout the organization. Data governance ensures that data can be trusted, and that people are held accountable for its quality. As an organization begins the data governance journey, it is important to take into consideration the following:

  1. Acknowledge Differences
    Stakeholders must recognize the differences between making a business case for data governance, and making one for a traditional technology project.  Many of these differences revolve around the non-conventional topics of business process, change management, business benefit, and holistic impact.  Understanding the differences will help shape your approach.
  2. Define and Clarify
    Take the time to describe what data governance is and what it means to your association.  A simple definition and several examples, together with an explanation of its purpose will go a long way toward getting everyone on the same page. Fine-tuning the characterization of this term and its intent will help align the staff and advance the dialogue with business leadership.  Make sure the purpose of the data governance initiative is documented.
  3. It’s All in the Name
    What you call something can determine how people perceive and respond to it – and whether they support it.  The term ‘data governance’ makes perfect sense to many in the information management space, but may not resonate with business leaders and executives.  For some, the word “data” represents a tactical responsibility of IT, while governance signifies bureaucracy. Neither of these invokes the sense that it is a cross-functional management activity.  You may want to consider alternate terms that better reflect the overall objective and are more likely to gain acceptance, such as “information asset management”.
  4. Set Expectations
    Identify short- and long-term expectations for the data governance program at both a business and technical level.  People involved in day-to-day information management activities have very different expectations than business leadership.  Examine realistic expectations that will satisfy both decision-makers and other stakeholders, such as staff and volunteer leadership.
  5. Value Proposition
    Determine whether the value proposition for the program will be based on a single project, the association as a whole, or somewhere in between.  While a specific project is important, a business case for data governance is more compelling when it addresses the impact across an entire data domain and enterprise.
  6. Interview the Business
    Determine whether the creation of the business case should involve one-on-one interviews with a representative from each of your association’s different departments.  This is where much of the business value can be found.
  7. People to Influence
    Draft a list of individuals and groups that will need to be convinced in order to secure funding and support for the data governance program.  Know who to persuade and why.  Never assume that people, even those involved in daily data activities, understand the benefits.
  8. Outside Assistance
    Decide whether the organization will enlist the assistance of a third party.  Doing so can add value in a number of ways:

    • Provide industry experience and expertise
    • Overcome the common stigma of being too close to the situation
    • Deliver the message to business leadership from an objective, non-partisan source
  9. What does Success Look Like?
    Establish the initial success criteria for the program.  The organization should know whether or not the program has been successful – you can’t manage what you can’t track.
  10. High-level Project Plan
    Establish an initial project plan with timelines to help identify and track activities and milestones.

While data governance may not be quite as exciting as data visualization, it is essential in order to maintain and leverage one of your association’s most valuable assets – your data. Initiating and executing a data governance program ensures that your organization and your customers will be served at optimal levels.

The Benefits of SQL Server 2014 for Your Association

Microsoft will be releasing SQL Server 2014 on April 1st and they will do so knowing that SQL Server is the most installed database in the world.  Additionally,  the amount of data being collected worldwide and waiting to be processed (or currently being processed) is growing tenfold every five years.  With such an enormous installation base, it is only a matter of time until customers adopt this version of SQL Server and take advantage of two of the most important new features targeted at addressing the massive amounts of data. SQL Server 14 for Associations
Speed:
All database developers have at some point come across a table (or a set of tables) so large that it requires waiting for data to load over an uncomfortably-long period of time. Unfortunately sometimes the user experience is even worse.  However, Microsoft has improved query performance by implementing an “In-Memory Columnstore.”  Here are some highlights of how your association can benefit from this feature:

  • It reduces the amount of data that needs to be processed. This is thanks to the new ability to look at specific columns, instead of using the traditional method which only allowed you to look at data by rows
  • It puts the data into memory – which is faster
  • Data is compressed up to 10 times and can also be indexed when stored in the Columnstore
  • Example: “By using In-Memory Columnstore, we were able to extract 100 million records in 2 or 3 seconds versus the 30 minutes required previously.”   – Bank of Nagoya

Hybrid Platform:
Database availability and scaling are critical to association professionals accessing data on the go and managing the daily increase of data storage requirements. The convergence of on premise installation and the cloud-enabled features of SQL Server 2014, such as cloud backup and cloud disaster recovery create a new hybrid platform configuration. This hybrid platform is designed to reduce costs, improve business continuity and simplify your recovery plan. If you decide to completely move to the cloud, there is a cloud migration wizard to transfer your entire SQL Server to the Microsoft Azure cloud.
It will be extremely interesting to see if the adoption of this version of SQL Server will shift the current cloud implementation trend dramatically.  For cloud providers, the new Columstore feature should help them provide faster data access, which will allow premise-based implementations to benefit.  Install, query, benchmark and share your results!