Archive for Business Intelligence

How Associations Are Successfully Using Artificial Intelligence

With AI no longer science fiction, associations are using advanced technologies to convert mountains of data into actionable insights.

At the recent EMERGENT event, hosted by Association Trends, we had the opportunity to jointly present case studies with ASAE’s Senior Director of Business Analytics, Christin Berry.

These success stories include how ASAE has:

Combined artificial intelligence and text analytics to enhance customer engagement, understand evolving trends, and improve product offerings

DOUBLED online engagement with unique open and the click-to-open rates using AI to personalize newsletters

Reduced the need for surveys, identified what’s trending, and measured through Community Exploration

Leveraged Expertise Search and Matching to better identify experts and bring people with similar interests together

I’m Matt Lesnak, VP of Product Development & Technology at Association Analytics and I hope to demystify these emerging technologies to jumpstart your endeavors in association innovation.

Text and Other Analytics

Associations turn to analytics and visual discovery for answers to common questions including:

  • How many members to we have for each member type?
  • How many weeks are people registering in advance of the annual meeting?
  • How much are sales this year for the top products?

Questions about text content can be very different, and less specific.  For example:

  • What is it about?
  • What are the key terms?
  • How can I categorize the content?
  • Who and where is it about?
  • How is it like other content?
  • How is the writer feeling?

It is widely estimated that 70% of analytics effort is spent on data wrangling.

This high proportion is no different for text analytics but can be well worth the effort. Text analytics involves unique challenges including:

  • Term ambiguity: Bank of a river vs. a bank with money vs. an airplane movement
  • Equivalent terms: Eat vs. ate, run vs. running
  • High volume: Rapidly growing social data
  • Different structure: Doesn’t really have rows, columns, and measure
  • Significant data wrangling: Must be transformed into usable format

Like the ever-growing data from association source systems that might flow to data warehouse, text content of interest might include community discussions, articles or other publications/books, session/speaker proposals, journal submissions, and voice calls or messages.

Possible uses include enhancing your content strategy, providing customized resources, extracting trending topics for CEOs, and identifying region-specific challenges.

Learn More

 

Personalized Newsletter

ASAE is working with rasa.io to automatically identify topics of newsletter content as part of a pilot that significantly improved user engagement.  ASAE and rasa.io first tracked newsletters interactions over time to understand individual preferences and trending topics.  Individuals then received personalized newsletters based on demonstrated preferences.

The effort had been very successful, as unique open and the click-to-open rates have more than doubled for the personalized newsletters.

Underlying technology includes Google, IBM Watson, and Amazon Web Services; combined with other machine learning tools developed by rasa.io.


Community Exploration

ASAE leverages a near-real-time integration with over 10 million community data points combined with enterprise data warehouse to analyze over 50,000 pieces of discussion content and over 50,000 site searches.  The integration is offered as part of the Association Analytics Acumen product through a partnership with Higher Logic.

Information extracted includes named entities, key phrases, term relevancy, and sentiment analysis.  This capability provides several impactful benefits.

Quick wins:

  • Visualize search terms
  • What’s trending
  • Staff and volunteer use
  • Reduce need for surveys

Longer-term opportunities:

  • Aboutness of posts as content strategy
  • Identifying key expertise areas
  • Connecting like-minded individuals

Underlying technology includes AWS Comprehend, Python, and Hadoop with Mahout.

Learn More


Expertise Search and Matching

Another application of text analytics that we’ve implemented involves enabling associations to better identify experts and bring together people with similar interests.  In addition to structured data from multiple sources, text from content including meeting abstracts and paper manuscripts provides insights into potential individual interests and expertise.

This incorporates data extracted from content using approaches including content similarity, term relevancy, validation of selected tags, and identifying potential collaborators.

Underlying technology includes Python and Hadoop with Mahout.


Approaches and Technology

We’re written extensively about the importance of transforming data into a format optimized for analytics, such as a dimensional data model implemented as a date warehouse.

Thinking back to the common association questions involving membership, event registration, and product sales; these are based on discrete data such as member type, event, and day.

Text data is structured for analysis using a different approach, but fundamentally similar as each term is a field instead of, for example, a member type table field.

Picture a matrix with each document as a row and each term as a column.

This is referred to as “vector space representation”.  With thousands of commonly used words in the English language, that can be a big matrix.  Fortunately, we have ways to reduce this size and complexity.

First, some basic text preparation:

  • Tokenization – splitting into words and sentences
  • Stop Word Removal – removing words such as “a”, “and”, “the”
  • Stemming – reduction to root word
  • Lemmatization – morphological analysis to reduce words
  • Spelling Correction – like common spell-checkers

Another classic approach is known as “Term Frequency–Inverse Document Frequency (TF-IDF)”.  We use TF-IDF to reduce the data to include the most important terms using the calculated scores.  TF-IDF is different from many other techniques as it considers the entire population of potential content as opposed to isolated individual instances.

It is widely estimated that 70% of analytics effort is spent on data wrangling.  This high proportion is no different for text analytics but can be well worth the effort.

Other key foundational processing:

  • Part-of-Speech Tagging: Noun, verb, adjective
  • Named Entity Recognition: Person, place, organization
  • Structure Parsing: Sentence component relationships
  • Synonym Assignment: Discrete list of synonyms
  • Word Embedding: Words converted to numbers

The use of Word Embedding, also referred to as Word Vectors is particularly interesting.  For example, the word embedding similarity of “question” and “answer” is over 0.93.  This isn’t necessarily intuitive and it is not feasible to manually maintain rules for different term combinations.

A team of researchers at good created a group of models known as Word2vec that is implemented in development languages including Python, Java, and C.

Here are common analysis techniques:

  • Text Classification: Assignment to pre-defined groups, that generally requires a set of classified content
  • Topic Modeling: Derives topics from text content
  • Text Clustering: Separating content into similar groups
  • Sentiment Analysis: Categorizing opinions with measures for positive, negative, and neutral


Finding and Measuring Results

With traditional data queries and interactive visualizations, we generally specify the data we want by selecting values, numeric ranges, or portions of strings.  This is very binary – either the data matches the criteria, or it does not.

We filter and curate text using similarity measures that estimate “distance” between text content.  Examples include point-based Euclidean Distance, Vector-based Cosine Distance, and set-based Jaccard Similarity.

Once we identify desired content, how do we measure overall results?  This is referred as relevance and is made up of measures known as precision and recall.  Precision is the fraction of relevant instances among the retrieved instances, and recall is the fraction of relevant instances that have been retrieved over the total amount of relevant instances.  The balance between these measured is based on a tradeoff between ensuring all content is included and only including content of interest.  This should be driven by the business scenario.

This overall approach to text analytics is like that used for recommendation engines based on collaborative filtering driven by preferences of “similar” users and “similar” products.


APIs to the Rescue

Fortunately, there are web-based Application Programming Interfaces (APIs) that we’ve used to help you get started.  Here are online instances from Amazon and IBM for interactive experimenting:

This is a lot of information, but the takeaways are they there are big opportunities for associations to mine their trove of text data and it is easy to get started using web-based APIs to rapidly provide valuable insights.

Learn More

 

Matt Lesnak, VP of Product Development & Technology
Association Analytics

Coolest New Power BI Features Revealed!

Recently the Microsoft Business Applications Summit 2019 highlighted new Power BI features and these are the coolest features to note IMO:

1. New Power BI App Workspace Experience in Preview Power BI App Workspaces were introduced to enable collaboration amongst the data/business analysts within an organization. The new experience introduces numerous improvements to better enable a data-driven culture including:

•   Managing access using security groups, distribution lists, and Office 365 Groups
•   Not automatically creating an Office 365 group
•   API’s for Admins, as well as new tools for Power BI Admins to effectively manage workspaces

2. Printing Reports via Export to PDF
You can now easily print or email copies of reports by exporting all visible pages to PDF.

3. Bookmark Groups
Now you have a way to organize bookmarks into groups for easier navigation.

4. Python Integration in Preview
Now data scientists can use Python in addition to R within Power BI Desktop.

5. New Visual Header
More flexibility and formatting options have been added to the header section of each visual.

6. Tooltips for Table and Matrix Vizs
Report page tooltips are now available for the table and matrix visuals

7. Many to Many Relationships in Preview
You will now be able to join tables using a cardinality of “Many to Many” – prior to this feature, at least one of the columns involved in the relationship had to contain unique values.

And now I’ve saved the best for last!

8. Composite Models in Preview
With this feature, you’ll now be able to seamlessly combine data from one or more DirectQuery sources, and/or combine data from a mix of DirectQuery sources and imported data. For example, you can build a model that combines sales data from an enterprise data warehouse using DirectQuery, with data on sales targets that is in a departmental SQL Server database using DirectQuery, along with some data imported from a spreadsheet.

As you can see there are many new features to digest but it would be well worth your while to follow the links provided.

On a closing note, I’d like to give you a teaser for two new features coming up soon that will have a huge impact on self-service data prep and querying for big data:

  • Dataflows
  • Aggregates

Stay tuned!
Mario Di Giovanni, BASc, MBA, CBIP
Director, Business Analytics

More about Mario

 

Moving the Needle on Member Data at NCACPA

Nikki Vann, CPA, is the Director of Finance & Administration at the North Carolina Association of CPAs (NCACPA).  She, along with Jennifer Rowell, Director of Member Engagement (also from the NCACPA) and I were honored to deliver a presentation entitled, “Moving the Needle on Member Data” at the American Institute of CPAs (AICPA) Annual Conference in Key Biscayne, Florida during July 2017.

“Our journey with business intelligence started as our Board discussed our strategic priorities,” Nikki explained. “They realized we couldn’t move forward with any of them without data. To better serve our members and their needs, we needed to understand the story of their actions through data.  Most recently, our team created 12 operational goals, and decided how to measure them.  Now we go through them at the staff meeting and we tie our efforts to those goals. We have made so many improvements to the way we use data as an asset.  For example, we used to go through the entire budget process with our executive committee and talk about each line item.  We realized we were dealing with highly intelligent leaders, so instead we decided to talk about the reason why we built the budget the way that we built it. Let’s talk about the changes we are making to how we do things because the data is telling us to.  Now at every board meeting, this is what they want. “

I see this trend continuing in high performing associations – the time that used to be spent pulling all the data together, can now instead be spend on deciding what to do about it and taking action.  Plus we all know the saying, what gets measured, gets done.   The idea is to use data to set a goal, then make a plan, and use data to measure results.  As Nikki says, “There is no success if you cannot measure it, and if it’s not quantifiable.”
One of the other significant accomplishments demonstrated by NCACPA during the presentation is the ability to visualize and analyze member engagement.  Jennifer added, “We are proud that during our work with Association Analytics we have connected our Abila AMS data with Higher Logic to get a wider view of what is important to our members, and what they are talking about.  We now have more confidence when we make decisions, and we’re also making better decisions because of it!”
 
 

Resolve Power BI Column Discrepancy

During development or enhancements to a Power BI solution, you may encounter this message: “An error occurred while processing the data in the dataset. The ‘handouts_path’ column does not exist in the rowset.” You probably received this message via email if you published the dataset.
powerbi_column_discrepancy

What Column Discrepancy Means

The message means that since the dataset was published, the actual data source (SQL Server, Redshift, Excel, etc.) was changed to remove or rename the column. Now, the published data source can no longer find it.
 

How to Fix Column Discrepancy

Here are the few steps to resolve the error message:

  1. Open the PBIX file in Power BI Desktop.
  2. Locate and click the refresh icon in the toolbar, as shown below.
  3. Wait for the refresh screen to complete. This will vary in length depending on the size of the data source.
  4. Once complete, confirm the change in the fields pane to the right. Any field removed should no longer appear and any field renamed should reflect the new name.
  5. Examine existing visualizations and correct any issues resulting from the changed field.
  6. Save and publish the package choosing the replace existing option when prompted.

powerbi_column_discrepancy_refresh
Monitor your inbox to ensure the issue is resolved by the elimination of the email indicating an error in the rowset. Now you’re ready to get back to analyzing the data.

Why You Should Include Data Literacy in Onboarding

A critical part of onboarding involves introducing the culture and setting expectations. If your association strives for an analytical culture that makes data-guided decisions, you should think about how to include data literacy in your onboarding or orientation process.

Data Literacy

Data literacy is the ability to derive and communicate meaningful information from data. Data literacy is an important skill for new employees to learn to make sure they know what data is available, where they can go to get answers to their questions, and how to interpret data. These competencies are essential in a culture that values an analytical mindset.

Levels of Data Literacy

The level of data literacy required depends on the roles and responsibilities of the new employee.
If they are heavily dependent on data, like a Director of Marketing, they should be able to interpret data, analyze data using business intelligence tools, identify key data sources, and communicate the results of analysis.
To maintain and strength a data analytical mindset within your association, all employees should understand the importance of evidence-based decision making. They should also be able to interpret meaning from the organization’s dashboards, visualizations, or reports representing the association’s key metrics for success.

Incorporate Data Literacy into Onboarding

Here are a few ways you can incorporate data literacy into your onboarding process:

  • Provide new employees with an orientation to all key data systems
  • Provide a tour of dashboards and visualizations your association uses
  • Encourage usage based on staff role and responsibilities. Set expectation for usage and monitor accordingly.
  • Document analytics decisions and processes and share with new employees.

A quality onboarding experience sets the stage for new employees to be successful and productive. Studies show the importance of onboarding. Did you know that newly hired employees are 58 percent more likely to still be at the company three years later if they had completed a structured onboarding process? Make sure your new employees are data literate and ready to make data-guided decisions. They’ll be more successful and satisfied in their new jobs and that’s good for everyone.

Workbook Performance Recording – Tableau Server

The first and most important rule about making workbooks more efficient is to understand that if it loads slowly in Desktop on your computer, then it will be slow on the server too once it is published. Desktop and server each have their own way to enable, record, and analyze performance. The focus here is on performance recording for workbooks published to Tableau Server.

Enabling Tracking

  1. Administrators must enable the feature. This is located under settings, for each site.
  2. Check the box and save for Workbook Performance Metrics.
  3. It is a good idea to leave this disabled when you are not using it since recording metrics can also impact performance.

tableau_performance_enable

Create the Recording

      1. Navigate to a view on the server.
      2. Remove the iid=xx from the URL.
      3. Enter in its place record_performance=yes. Your full URL should now look something like this: https://data.associationanalytics.com/#/site/AA/views/AAEmailActivity/MessageStatisticsSummary?:record_performance=yes
      4. After the page reloads, you’ll notice the ID is added automatically back to the URL and that a performance button appears within the View’s toolbar. Don’t click on the performance button yet.
      5. Do some filtering and some clicking within the workbook such as applying filters, selecting marks/rows, and clicks that cause actions to other elements of the visualization.
      6. Then click the performance button.
      7. Now you’re ready to click on the Performance button which will launch a new window with the collected statistics (see next image).
      8. The analysis and follow up actions are a whole other topic, but to quickly mention that you want to make sure your timeline slider is all the way to the left and then you’ll be able to see the different events and which takes the longest: executing query, sorting data, building view, connecting to the data source, geocoding, or computing layout.
      9. The provided workbook is not directly sharable, but the capability to download the resulting workbook is provided. Further, it is possible to use the download to publish it to another location.
      10. Don’t forget to disable the performance recording in the admin settings when you are finished.

tableau_performance_viz
Now that you know you can record and view the results of a workbook published Tableau Server, you can start to analyze the results so they load faster. In separate posts we’ll cover performance recording in Desktop and how to interpret the provided visualization.

Output vs. Outcome: Metrics for Email Marketers

Email marketing is a primary way that associations engage with members. With the cost of email marketing systems and staff time, the investment in email marketing can be significant. So how do you measure the success of emails to ensure a return on investment?
There are two types of email metrics or key performance indicators (KPIs) – those focused on Output and those focused on Outcomes. What’s the difference? Output is the “how” — how you do something or, in this case, deliver an email. Harvard Business Review defines outcomes as “the difference made by the outputs” or “the benefit your customers receive.”

Output Email Metrics

Output Email Metrics focus on operational performance, which is generally measured by bounce rate and type of bounce. This is more about the quality of the list and a good barometer for data hygiene.

  • Bounce Rate = (Number of Bounces/Total Recipients of Email Campaign) * 100
  • Type of Bounce (e.g., Hard Bounce, Soft Bounce, etc.)
  • Spam Complaint Rate = (Number of Spam Complaints/Total Recipients of Email Campaign) * 100
  • Unsubscribe Rate =  (Number of unsubscribes/total recipients) * 100
  • Churn Rate = Percent change in list size after the unsubscribes, complaints, and hard bounces are taken into account

These are important metrics to track as they can help identify data quality and deliverability issues.

Outcome Email Metrics

Outcome Email Metrics focus more on engagement and the effectiveness of your email campaign at convincing readers to take action.

  • Conversion Rate = (Number of people who completed the desired action/Total Recipients) * 100. Conversion rate is the best metric for measuring outcomes of email campaigns. Here’s a tutorial on setting up Google Analytics to measure conversion rates from Smart Insights.
  • Open Rate = Emails Opened/(Emails Sent-Number of Bounces) * 100
  • Clickthrough Rate = (Total clicks OR unique clicks/Total Recipeints) * 100
  • Email Sharing/Forwarding Rate

There are many email metrics that you can use, but, ultimately, metrics should be based on your strategy and unique goals.
Banner Designed by Freepik

Using Learning Analytics to Support Students

For many associations, education is more than just a source of revenue and a key member benefit. It is a cornerstone for the organization’s existence. Despite the importance of education, associations still struggle with how to measure the success of professional development programs. According to Tagoras, 87% of associations offer e-learning, but less than 30% use data to make decisions about their educational programs.
The problem is two-fold for associations.
First, there’s the question of what associations should be measuring to gauge the impact programs have on students.
Second, associations have to figure how to measure impact.

What to Measure in Learning Analytics

Recently, Debbie King, Association Analytics® CEO, presented a session about learning analytics at the American Society of Association Executives (ASAE) Spark Conference, an online conference about the art and science of adult learning. What is learning analytics and how can it help your association?
George Siemens, a noted expert on the topic, wrote that “learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimizing learning and the environments in which it occurs.”
This is distinct from business analytics for educational programming, which focuses on operational or financial performance.
Most associations have access to business analytics for their educational programming. They can find revenue, expenses, demographics of registrants, or number of registrants for any given program. This is important information, but doesn’t necessarily provide insight into how to better engage students or how to improve student success.
To better understand the distinction between learning analytics and business analytics, it’s helpful to look at the difference between common key performance indicators in both areas.
Learning Analytics KPIs
See the difference? Learning analytics is focused on the student and their success. Business analytics for professional development is focused on the program and its success. The two are closely related, but separate and its worth considering both to ensure program success.

How to Measure Learning Analytics

One major obstacle to engaging in learning analytics is determining how you will measure professional development programs.
Where is the data needed to understand impact? How do you combine data from multiple, disparate data sources to easily analyze information in a single location?
After you define what you want to measure, you need figure out where to get the information. Relevant data sources may inclue your Learning Management System (LMS), Association Management System (AMS), Event Management System (EMS), and even yoru web analytics program.
The data can then be integrated with a data warehouse where it can be combined and prepared for consumption by business users. You can visualize and interact with the data using a business intelligence tool, like Power BI or Tableau. To learn more about data warehousing, see our post on data warehousing.
Banner Designed by Freepik

Customer Experience is about the Journey

The constantly growing number of choices for customers mean many aspects of our products and services are in danger of becoming commoditized, but customer experience is not one of them.

Importance of Customer Experience

Gartner reports that “By 2017, due to internet-enabled price visibility, the digital customer experience will be the key differentiator of your organization.”
But it’s not only digital — customers expect consistent, seamless, high value experiences – both digital and in-person. If you don’t give it to them, they will go somewhere else.

Providing Outstanding Customer Experiences

How can associations continue to provide outstanding customer experiences to remain relevant to members and prospects?
Similar to commercial brands, associations have to view every interaction through the lens of the customer.  Sounds simple, but its easier said than done.
At a minimum, associations have to understand and manage their touchpoints. Customer touchpoints are your brand’s points of customer contact. Examples include online advertising, social media, direct emails, customer service, events and conferences, membership joins and renewals, purchases, and surveys.
It’s important to carefully manage each point of interaction. A single negative touchpoint can sour the customer on the overall experience.

Customer Journeys

Interestingly, the research shows that careful attention to individual touchpoints may not be enough. The customer experience with your association happens over a period of time, across multiple channels, and most importantly, across multiple business functions. These multichannel, cross-functional interactions are called Customer Journeys.
Consider a customer registering for an event.
In this example, the customer navigates through multiple channels (print, website, direct mail, word of mouth, social media, in person).
In addition, their touchpoints span multiple departments (Marketing, PR, Events, Publications, Member Services, Finance). Each department has different goals and performance measures for their customer interactions. The challenge is to move away from this siloed approach to managing customer interactions.
When we think of the journey instead of just individual moments, we can see each touchpoint influences the others and the whole journey is greater than the sum of its parts. This cross-functional, customer-centric view is the essential first step to analyzing and improving your customer’s journeys.

Customer Journey Analytics

To understand customer journeys, you need Customer Journey Analytics – analyzing how customers use the available channels and touchpoints to interact with our organizations. You can read more about Customer Journey Analytics in Association Analytics® News and subscribe to our monthly newsletter for more information on the latest trends in nonprofit data analytics.

Associations Tableau User Group Covers Best Practices in Visualizations and Advanced Analytics

The Associations Tableau User Group met in Chicago on October 13, 2016. Formed in August 2016, the Associations Tableau User Group (AssocTUG) serves as a platform for association professionals using Tableau to network and learn.  The group meets quarterly in Washington, DC and periodically in Chicago. If you missed the Chicago meeting, here’s a recap of some key takeaways.
Association Analytics® was proud to sponsor the meeting in Chicago. Association Analytics® co-founded AssocTUG and currently serves on the planning committee for AssocTUG.

Best Practices in Visualizations

Matthew Illuzzi reviewed some best practices in visualizations. Don’t just recreate a spreadsheet in Tableau. When developing a visualization consider your audience and what will resonate with them. Focus on telling a story. If you’re feeling lost, Tableau offers a ton of great educational resources.

Analytics for the 99%

Christopher Michaelson, Data Analytics Manager at the National Futures Association, shared how he introduced Tableau and data analytics to his organization. His goal was to reach “self-service nirvana” where everyone had access to data analytics and used data to make decisions.

Getting Started

To start, NFA purchased only a few Tableau Desktop licenses. They used Tableau Reader, which is free and allows users to open packaged Tableau workbooks, to give staff access to visualizations. Chris distributed packaged workbooks on SharePoint. Chris noted that managing multiple workbooks this way can be difficult to manage. He recommended considering Tableau Server licenses when managing multiple workbooks in this way is no longer a viable option. NFA publishes data sources to the server now, but eventually wants to get a data warehouse.

Designing Visualizations

Michaelson recommended focusing on creating beautiful and simple designs. Less is more in data visualizations. He also focuses heavily on usability and looks at minimizing how many clicks it takes to get an answer. He leverages Tableau strengths. Tableau is not designed to handle tabular views. Echoing Illuzzi, Michaelson warned against trying to recreate Excel in Tableau.
Most importantly, Michaelson said that information presented in data visualizations has to be timely and relevant. Does the data visualization solve a problem or create more work? Would stakeholders adopt the data visualization voluntarily? The fastest way to spark interest is to build visualizations that people will want and need.

Matt Lesnak presenting at AssocTUGAdvanced Analytics and Tableau

Did you know that you can use Tableau for advanced analytics, like propensity modeling and predictive analytics, by connecting it to the statistical computing tool R? Association Analytics® Senior Analytics Architect Matt Lesnak provided an overview for how you can connect R to Tableau to handle advanced analysis. Lesnak shared a demonstration of how he used a model created in R to populate a data visualization in Tableau that used unstructured data from social media.

Get Involved with Associations Tableau User Group

Don’t miss the next AssocTUG meeting on November 16, 2016, 1:30-4:30 p.m. at the National Council of Architectural Registration Boards (1801 K St NW, Suite 700K, Washington, DC 20006). Register and learn more.
Join the AssocTUG online community to stay up-to-date on the latest group activities and connect with other Tableau users in the association community.