Read Me First!
      Back to home
      1. Metolius Knowledge Base
      2. Getting started
      3. Read Me First!

      5 Things to Consider When Developing Surveys for Metolius

      Learn key factors for creating surveys in Metolius. Explore best practices for effective survey development.

      Table of Contents

      Different by Design

      Metolius is different from any other survey platform you have used. It is different by design and was purpose-built to chart a new paradigm for survey analysis. But to harness the full power of Metolius, you need to first understand the survey structure that allows Metolius to create data analysis immediately. This requires planning and understanding how your questions will support your desired analysis.

      Metolius leverages a longstanding methodology of survey construction: the Intelligent Questionnaire (IQ).  Both the philosophy and method of the IQ have been around for a few decades, but until Metolius, no survey platform has been designed to support and extract the powerful analysis an IQ can provide. By following a series of steps before you get started writing questions, you will create an insightful analytic survey. This "front-end" planning, will save you hours traditionally spent in data analysis on the "back-end" after your survey is complete and you need to understand its results. No more Excel exports, no more cross-tabs, and no more pivot tables.

      Don't be alarmed! Metolius is loaded with survey templates, but to harness the true power of the platform, it will be up to you to go off script from the templates and understand the needs of your organization, department, or team, to develop a truly bespoke solution to the problem you're looking to solve or the goal you're trying to achieve.

      The 5 concepts below are important to consider when creating an intelligent questionnaire in Metolius. 

      Goal Planning

      The ideal intelligent questionnaire begins with knowing the goal of your survey. Surveys can measure productivity, risk, customer satisfaction, employee awareness, performance, compliance, or dozens of other things.  Why are you writing this survey? What do you hope to learn? What decisions do you have to make? What do you want to explore? What information do you want to derive? What is the qualitative goal and how is that goal measured? Simply said, what is the yardstick necessary to provide you with your desired analysis?

      If you plan your survey before you begin to build it in Metolius, you'll be more aligned in how to measure and evaluate it, making analysis much more insightful and results-driven for your specific needs.
       
      These are the questions you need to ask yourself before you start writing questions. 
      • What is the purpose or goal of your survey or survey?
      • Who is the target participant? I.e.: Who is going to take your survey?
      • How would you like to sort or filter the data? E.g.: age, tenure, gender, location, regional office site, education level, etc.

      The terms 'intelligent questionnaire', 'assessment', and 'survey' are all used interchangeably throughout the Metolius Knowledge Base.

       

      Survey Structure

      Surveys written for Metolius follow a systematic structure. Three components are nested in support of the survey goal -Outcomes, Intentions, and Questions. 

      Most traditional survey instruments are created with the hope that the results will offer clear insights and actionable information. Unfortunately, without the forethought of breaking up your survey goal into smaller components, and the ability to quickly filter a structured set of questions, the actual analysis of the survey responses is generally insubstantial, time-consuming, and frustrating.  In developing an intelligent questionnaire, we encourage you to start with a goal of your survey and then think about what categories and components, and then sub-categories or sub-components would contribute to measuring that goal. 

      The first category or component breakdown would be called an ' Outcome' in Metolius. The secondary sub-category or sub-component breakdown from that Outcome would be called the ' Intention' in Metolius. Within the Intentions is where your questions will live. Below is a further explanation of these component breakdowns that make the structure of a Metolius Intelligent Questionnaire. The more forethought put into the categories listed below lends to better analysis and how well you can 'slice-and-dice' your data.

      Outcomes

      What is the highest level of necessary categories or components, that should be present to measure and understand your goal? These over-arching categories or components of your goal are what we call 'Outcomes'.
      Most survey instruments are looking for a single outcome to understand a goal. But, some goals require measuring a combination of separate-yet-critical components; aka 'Outcomes' as they're called in Metolius. Based on the goal of your survey, you will need to select a single or multi-outcome survey type to support the measurement of your goal. If you require only a single outcome for your survey in Metolius, you can do that. It can be synonymous with your goal. You can make your surveys as simple, or as robust, as you like in Metolius.

      Single Outcome

      Single Outcome Structure
      A single outcome survey is the most common. They contain several intentions, and each is made up of supporting practices (questions). For example, if the survey goal was to understand customer satisfaction for a new product, then this could be the single outcome, where you would only have a breakdown of the sub-components, aka the 'Intentions'. For this example, let's say the product was for a child's toy. Intentions in this example could be price, quality, ease of use, and fun. Do you see how thinking about what your goal is can help build the survey's structure?

      Multi-OutcomeMulti-Outcome Structure

      A multi-outcome survey supports more complex processes and concepts. These surveys require identifying several outcomes (minimum of 2) that make up or support, a complex goal. For example, if the goal of the survey was to analyze the quality of a distribution process, then the outcomes would likely include line speed, quality control, and unit throughput. Or, say you are surveying internal stakeholders in your business with the goal of better understanding your company's health. To get a full 360 view of your business, you know you need to evaluate revenue, marketing, distribution, and people.  In both examples, establishing those Outcomes in your survey supports evaluation and drill-down into further sub-categories or sub-components, aka 'Intentions'. 

      Intentions

      For each of the Outcomes that support your goal, what are the actions that need to be present that validate each of them? These supporting actions, or sub-components or sub-categories, are what we call 'Intentions'. 

      In essence, you have a goal, and you have identified the Outcomes (over-arching components) that need to be present to achieve the goal.  Now, you need to think about the supporting activities or 'Intentions' that are needed to achieve the Outcomes you have identified. From here, you will build your questions.

      For example, if the Outcome is employee performance-based, how do you measure their performance? What indicators must be present? For example, if you were evaluating a Sales Department and your Outcome is performance, the Intentions could be revenue, customer satisfaction, company resources, and work environment. The higher the performance level or quality of these Intentions, all contribute to the success, or shortcomings, of the desired outcome of performance in this example.

      Questions

      Thinking down one more level are the actual questions that will make up your survey. These are what we call 'Practices'.

      When you consider the Intentions that are necessary for the success of an Outcome, the best practices, activities, or visible elements that are necessary for each intention to succeed should make up your questions.

      These are now the questions that make up your survey.  Using the example above, questions supporting revenue performance might follow this logic:

        • How competitive is our pricing versus the overall market sector? (very competitive to not competitive)

        • How effective do you feel our advertising is in bringing sales opportunities forward? (very effective to not effective)

      These types of questions explore the tactical areas that contribute to the success or failure of the Intention.

      Once you have these organized, Metolius does the rest.  It will automatically analyze your responses and provide you with graphical feedback that is well-organized and easy to understand. Insights into the responses will be natural and you can spend more time thinking about and understanding the information, rather than hours fighting the data.

      Rating Scale

      Most surveys utilize a 5-point scale of measurement, with 5 being the highest value.  

      But what are you measuring?  For example, let's assume the goal of your survey is to evaluate the team's impression of the current state of readiness in a production environment.  The first determination necessary is defining the desired level of readiness.  Simply said, what does success look like?

      Using a 5-point scale, this level of readiness would be a 5 but how do you describe the other tick marks of the scale?  What defines your other levels?

      It is up to you to define a set of levels from unacceptably low (1), to needs improvement (3), to perfection (5).

      Ask Easy and Direct Questions

      When you consider your definition of the desired level of readiness, what attributes or objectives need to be present and high functioning?  These objectives should be actions, processes, or procedures that are necessary for success and may represent best practice examples. The questions associated with these objectives should identify the practice steps to achieve success and evoke the observation, awareness, or functionality of these practice steps from the survey participant.
      For example, a practice might be monitoring production limits.  
      The appropriate question could be:
      • Our team monitors the production limits as frequently as necessary to maintain maximum product throughput from our operations.
      The related response option might be:
      • Strongly agree, Agree, Neutral, Disagree, Strongly disagree

      Avoid double-barrel questions as they refer to two or more separate issues or topics, but the question can only have one answer. 

       

      Scoring Criteria

      When adding numerical values to your question responses, Metolius offers the ability to choose how that question is calculated.  This is because statistical surveys require adjustments to the data for negative responses. Metolius has eliminated the need for this data manipulation as each question can be scored as positive, neutral, or negative allowing your cumulative score to be calculated automatically and not require a post-survey adjustment; To note,  the survey author is required to identify these types of questions at the time they are created. There are three response categories. The default category for all questions is Positive.
      1. Positive - Choosing a positive response value will add a positive number to the score and affect the score positively; Effectively increasing the overall value.
      2. Neutral - Choosing a neutral response value neither adds nor subtracts a number from the score and the overall value is not affected.
      3. Negative - Choosing a negative response value adds a zero to the score; Effectively reducing the overall value.
      When you select the neutral category, this response will not be added to the overall calculation toward a composite survey value. Most often it is applied to the ‚"I don't know" response. In the neutral setting, if someone selects "I don't know" it does not contribute to the composite survey calculation and value. When neutral is selected, a warning will pop up to confirm you do not wish to exclude that value from the final calculation.
      However, if your survey is written with the expectation that the survey participant should be able to respond to the question and‚ 'I don't know' reflects a deficiency; then you have the option to select the negative category for this response. With this setting selected the value of ‚ 'I don't know' (generally zero) will be calculated as part of the composite survey score, and the addition of a zero will effectively reduce the overall value.