1.4 Research Instruments, Measurement, and AI Tools

SurveysResearch tools used to elicit respondents’ reactions to specific questions. are one of the primary methods management researchers use to learn about OB. A basic survey involves asking individuals to respond to several questions. The questions can be open-ended or close-ended. An example of an open-ended question that could address your manager’s question would be to ask employees how they feel about goal setting and productivity, then summarize your findings. This might work if you have a small organization, but open-ended surveys can be time-consuming to summarize and hard to interpret at a glance. You could get more specific by asking employees a series of close-ended questions for which you supply a response key, such as a rating on a scale of one to five. Today, it is easy to create online surveys that automatically compile results. Several free survey tools are available online, or you can use paper-and-pencil surveys. Both approaches have pros and cons, but matching your survey technique to the particular sample and research questions is key. Some organizations gather a little bit of information frequently using pulse surveysA quick and easy method of collecting data using only a few questions.. Often, organizational opinion surveyA questionnaire designed to evaluate employee morale, engagement, and overall satisfaction. (also called an attitude survey) data is a popular way for organizations to hear from all their employees at least annually. Such surveys are widely used because they are great predictors of behavior and allow employees to voice concerns and compliments.

Regardless of the method you choose to collect your information, the next step is to look at the average of the responses to the questions and see how the responses stack up. But this still wouldn’t really answer the question your boss asked, which is whether using goal setting would help employees be more effective on the job. To do this, you would want to conduct a field study.

Field studiesResearch conducted in actual organizations. Studies may include observation, interviews, surveys, or experiments. are those conducted in actual organizational settings with a population of workers. Most field studies involve surveying employees and managers working in organizations. Field research involves studying organizational behavior in real-world settings, such as workplaces or communities. Researchers collect data through direct observation, participant observation, interviews, or surveys to understand organizational dynamics, interactions, and outcomes in their natural context. Field research allows for a holistic understanding of organizational behavior and can uncover nuances that may not be captured in experimental or laboratory settings.

Compelling evidence comes from field or lab studies that employ an experimental designA study including a group that receives a treatment and a comparison group that receives no treatment.. Here you would assign half the employees at your company to the goal-setting condition and the other half to the control groupA group that does not receive any experimental manipulation so it can be compared to a treatment group. condition. The control group wouldn’t get any information about goal setting, but the treatment groupA group that receives experimental manipulation. would. If you found the treatment group more effective than the control group, you could tell your boss that goal setting works.

OB researchers are often interested in basic research questions such as “Can we show that goal setting increases performance on a simple task?” This is how research on goal setting started and how we can establish the conditions under which it works more or less effectively. To address this, researchers may conduct a lab studyResearch conducted under controlled conditions (may include observation, interviews, surveys, or experiments)., which is a study conducted in artificial situations outside of actual organizations. Lab studies often follow an experimental design, which involves manipulating one or more variables to observe the effect on another variable. You may even have been involved in a lab study at your university. One of the most important concepts to understand about lab studies is that they give the researcher a great deal of control over the environment they are studying but do so in a less “realistic” way since they are not studying real employees in natural work settings. For example, in a lab study, a researcher might create an artificial job in which “employees” are paid for each T-shirt folded. Then, the researcher can examine the effects of goal setting by setting a goal for the experimental group and not setting goals in the control group, comparing the resulting performance levels. While such a study would be interesting for examining the effects of goal setting, it is a study on students acting as employees for a short period. It, therefore, would give us limited information about whether goal setting would work in actual organizations. 

Case studiesIn-depth descriptions of a single industry or company. are in-depth descriptions of a single industry or company. Case study research involves an in-depth examination of a single organization or a specific situation within an organization over time. Case writers typically employ a systematic approach to gathering data and explaining an event or situation in detail. Researchers collect qualitative data through interviews, observations, and documents to gain insights into organizational processes, behaviors, and outcomes. The benefits of case studies are that they provide rich information for drawing conclusions about the circumstances and people involved in the topics studied, as case studies provide rich, detailed descriptions of organizational phenomena and allow for in-depth analysis of complex issues. The downside is that it is sometimes difficult to use what worked in a single situation at a single organization and generalize what might work in other situations and organizations. 

Meta-analysisThe process of summarizing research findings from studies on related topics. is a technique used by researchers to summarize what other researchers have found on a given topic. This analysis is based on taking observed correlations from multiple studies, weighting them by the number of observations in each study, and finding out whether the effect holds or not. For example, what is the average relationship between job satisfaction and performance? Research shows that looking across 300 studies, the relationship is moderately strong. This is valuable information because, for years, people had thought that the relationship did not exist. Still, when all the studies were examined together, the original beliefs about the satisfaction–performance relationship deteriorated. The advantage of meta-analysis is that it gives a more definitive answer to a question than a single study ever could. The downside is that meta-analysis is only possible if sufficient research has already been done in a given area.

In addition to research methods and measurement, it is important to understand the different analytics approaches to answering key OB questions. There are three approaches to analytics: descriptive analyticsApproaches focused on understanding what has already happened. , which refer to approaches focused on understanding what has already happened, such as the average job satisfaction rating for employees from last year’s opinion survey; predictive analyticsA focus on what is likely to happen based on what we already know. , which refer to what is likely to happen based on what we know already, such as guesses regarding this year’s upcoming opinion survey ratings for job satisfaction; and prescriptive analyticsA focus on what should be done in the future based on what we know. , which refer to what should be done in the future based on what we know, such as specific evidence-based actions that might be taken to increase job satisfaction. 

Big dataThe volume, variety, velocity, and veracity, or validity, of data.  refers to the data’s volume, variety, velocity, and veracity, or validity. Companies like Facebook, Amazon, and Google track huge volumes of user and consumer data and use sophisticated algorithms to inform advertising and to predict what consumers will be interested in purchasing in the future based on their past browsing and shopping behavior. While finance, marketing, and operations management functions have used big data for a while, it is a relatively new concept for OB and Human Resource Management. Today, the best approaches for aligning OB questions utilizing such large volumes and varieties of data are studied by OB scholars regularly.

When it comes to AI technology, having all four V’s of big data is critical. An important thing to remember is the old adage “garbage in, garbage out.” While the scale of data (volume), different forms (variety), and streaming data (velocity) are all relatively easy to come by as processing power gets faster and more readily available to process vast amounts of data, the quality of the solutions based on AI processing can only be as good as the assumptions and data they are built upon (veracity). 

You may have heard about machine learning lately. The idea behind machine learning is that as large amounts of data (big data) have become more readily available, traditional approaches to data analysis are not feasible. Machine learning algorithms generate models based on the available data and continue to update and adjust as new information is available. Machine learning may also simulate what respondents might say or do. “It’s not magic,” Greg Corrado, a senior research scientist at Google, has said of machine learning. “It’s just a tool but a really important one.” When you use Netflix and notice it suggests things you might enjoy watching, that recommendation was based on machine learning. In that case, you’ve experienced an example of machine learning applying an algorithm based on your viewing history and that of other viewers. Machine learning relies on machine learning algorithms. So, what are those?

A machine learning algorithm (MLA)A computational method or procedure that enables computers to learn from data and improve their performance on a specific task without being explicitly programmed. is an AI technology that enables computer systems to learn from data and improve their performance over time without being explicitly programmed. These algorithms analyze large datasets, identify patterns, and make predictions or decisions based on the information they have learned from the data. Machine learning algorithms are used in various applications, such as image recognition, recommendation systems, and predictive analytics. They play a crucial role in artificial intelligence and enable systems to adapt and improve over time as they encounter new data. In organizations, machine learning algorithms can analyze large volumes of employee data to uncover patterns, trends, and insights that can inform decision-making and improve management practices. For example, these algorithms can predict employee turnover, identify high-performing candidates during recruitment, or personalize learning and development programs based on individual employee needs and preferences. By leveraging machine learning algorithms, organizations can optimize their human resource management strategies, enhance employee satisfaction and productivity, and ultimately achieve their organizational goals more effectively.

Natural language processing (NLP)A field of AI focused on enabling computers to understand, interpret, and generate human language in a way that is meaningful and useful. is a branch of AI that focuses on enabling computers to understand, interpret, and generate human language in a way that is meaningful and useful. NLP algorithms analyze and process text or speech data, allowing computers to extract insights, identify patterns, and make sense of human language. In organizations, NLP can automate various tasks related to managing employees. NLP plays a crucial role in various applications, including virtual assistants, chatbots, search engines, and text analytics, and it continues to advance rapidly, making human-computer interaction more intuitive and efficient. For example, NLP-powered chatbots can handle employee inquiries and provide real-time support, streamlining communication and improving employee satisfaction. NLP algorithms can also analyze employee feedback, performance reviews, and sentiment on social media platforms, helping organizations gain insights into employee attitudes, concerns, and engagement levels. By leveraging NLP technologies, organizations can enhance communication, streamline processes, and make more informed decisions.

A chatbotA computer program designed to simulate conversation with human users, typically through text-based interactions. is a computer program that simulates a conversation with human users, typically through text-based interactions. Chatbots use natural language processing and other AI techniques to process user queries, provide responses, and perform tasks. Chatbots can be deployed in various ways, managing employees to streamline communication, provide support, and automate routine tasks. Recently, chatbot use has exploded, with projections estimating their market is worth $5 billion. ChatGPT (OpenAI) and Gemini (Google) have become well-known. Organizations can use chatbots as virtual assistants to answer employee inquiries about company policies, benefits, or procedures in real time. Chatbots can also facilitate employee onboarding by guiding new hires through the process, answering questions, and providing relevant information. Additionally, chatbots can be integrated into collaboration platforms or HR systems to automate tasks such as scheduling meetings, submitting time-off requests, or accessing training materials. By leveraging chatbots, organizations can enhance employee engagement, improve efficiency, and provide round-the-clock support to their workforce. 

RoboticsA field of technology that involves the design, construction, operation, and use of robots. is a branch of technology that deals with robot design, construction, operation, and use. Robots are programmable machines that can carry out tasks autonomously or semi-autonomously, often mimicking human actions or performing repetitive, dangerous, or impractical tasks for humans. In organizations managing employees, robotics can be utilized in various ways to enhance efficiency, productivity, and safety. One way organizations use robotics to manage employees is through the automation of repetitive or labor-intensive tasks. For example, robots can be deployed in manufacturing facilities to assemble products, handle materials, or perform quality control inspections, freeing human employees to focus on more skilled or creative tasks.