Ask the Experts: The State of Data Exploration and AI

A recent survey found that 64% of organizations face a shortage of data science skills and that lack is holding back their data exploration. Upskilling business analysts with advanced analytics capabilities is key to gaining the competitive edge that comes from exploring complex data sets, but knowing where to start and what tools will speed up this journey isn’t always clear.

We turned to the experts on our Scientific Advisory Board for their insights on the foundational skills necessary for data exploration, how network science and AI fit in, what ChatGPT will really replace in the world of work, and more.

For this discussion, we’re joined by:

Each of these experts brings valuable insights directly to our teams, helping them develop the kinds of solutions that have earned patents and solved complex challenges. We’re excited to share their thoughts on some of the most common questions in data and analytics today. 

What can business analysts who are new to advanced analytics do to start building up their data exploration skills?

Robert: Data exploration works best when you have someone who is sophisticated in statistics working with someone just starting out. It’s like an apprenticeship. Also, analysts should just start playing with the data because oftentimes that leads you to the right questions.

Kostia: Formulating the right question is a key foundational skill. If you want to do intelligent analysis, at minimum, you should know the right question to investigate and which data set to look at for answers.

Adam: Agree. If you get data exploration wrong, you’re looking at the wrong problem. You’re missing the point. It’s the absolute foundational first step to any analysis.

On the other side of that coin, can experts like data scientists benefit from data exploration tools?

Adam: Definitely. Having a tool that surfaces counterintuitive things you’re not going to see unless you were looking for it isn’t something that experts will have off the shelf. 

Kostia: Data scientists should also look at a tool—like Virtualitics—as a kind of MATLAB for data exploration. Use it to try out something quickly and see if it could work before you spend the time to program it yourself.

Can building a dashboard for reports achieve the same outcomes as data exploration?

Adam: Dashboards still rely on the user of the tool to do the exploration with their own insights and skill. Whereas, if you can expose those insights in an automated way, that’s big. Dashboards just can’t do that. Data exploration requires better tools. If you have easier tools for visualization and highlighting, you’re going to see a broader set of topics in more depth than if you just let intuition guide you to where you think connections in the data might exist.   

What does network science bring to the table for data exploration that wasn’t there otherwise?

Adam: As a data analyst, if you ignore network structure, you’re ignoring hundreds of important features connecting things in the data that could lead to new insights and conclusions. And most tools give you nothing. You miss all of that. And so having a tool that can let you visualize and explore the graph structure is crucial.

Speaking of automation, AI is top of mind in every business. What types of research are critical to the future adoption of AI?

Adam: Causality and safety. By that I mean, trustworthiness and interpretability. There’s a lot of racing ahead on “I can do an okay job for lots of questions and lots of topics,” but to make AI deployable in meaningful ways, it needs to do more than an “okay” job. 

Robert: Long-term cognitive research is needed about what instills trust. I agree that everything is racing ahead so fast now with AI and large language models (LLM) that no one is thinking about human-computer interaction like they used to. When they first designed the predecessor to the Macintosh, they did all this research at Bell Labs to figure out what was the cognitive load of moving a mouse and where should the trash can go to minimize the load on humans. They really thought about how to interface with the human side of it and instill trustworthiness. But I don’t really see this yet with AI. 

Will tools like ChatGPT replace data analysts and other jobs?

Adam: The more likely use in the near- and long-term is that it will make us more efficient at the things we’re doing. It won’t be that it isn’t working with human input. It’s working with human input to understand and target usage around how to help you get started, go faster, and be more careful. For example, I don’t think we’re far from a ChatGPT-like plug-in being trusted and used for math calculations. Then, the data exploratory piece becomes even more important and this piece will remain a human’s job. 

Kostia: AI will definitely take some jobs out of the market, especially routine ones, but everyone needs to learn how to use AI and ML solutions. It’s like how everyone needs to learn to use a computer now. And did the computer eliminate some jobs? Yes, but in many ways they created much more.

So, how do we prepare young professionals for a future where AI might be as commonplace as computers and cell phones?

Robert: We should train them all in the standard philosophy of what ethics means and what’s been learned about that. Because everyone can learn about the algorithms, but the ethical questions run deeper. 

Kostia: Yes, for business leaders, AI ethics is even more crucial to learn than the mathematical part. They may not design the algorithms themselves, but they have to guide groups of people to do this. And for that, having strong training in ethics philosophy is important.

Adam: People have this view that AI is magic. People don’t understand how it works, but it solves their problems. They don’t get the limitations of the various tools or understand that you’ll get terrible results when you apply them to the wrong places. Knowing when and when not to apply AI is key. Teaching those limitations, ethics being one of them, is essential.

A space where innovation can thrive

Our Scientific Advisory Board plays a key role in our innovative culture at Virtualitics. Our goal is always to build an environment where scientists and engineers can collaborate together, solving challenging problems that will deliver positive impacts across industries. 

Sound like the kind of team you’d like to be part of? Take a look at our current job openings.

Related Articles

Virtualitics Wins Three 2024 Globee® Awards Including Asset Management Innovation

5 Ways AI Leads to Better Maintenance Strategies

Virtualitics Named One of America’s Best Workplaces 2024 by Inc. Magazine

SDC Exec

Maintenance Decision Intelligence AI Transforms Enterprise Asset Management

Virtualitics’ Maintenance Decision Intelligence AI Application Transforms Enterprise Asset Management

Why Implementing Decision Intelligence Requires Data Transparency

Virtualitics Wins Three 2024 Globee® Awards Including Asset Management Innovation

Recognized for New AI-Powered Maintenance Decision Intelligence Application, AI and ML Technology, and CTO of the Year PASADENA, Calif., July 8, 2024 — Virtualitics, a

5 Ways AI Leads to Better Maintenance Strategies

Maintaining a fleet of assets requires a nuanced understanding of what’s running smoothly—and what isn’t. A modern maintenance strategy must be data-driven, helping teams see

Virtualitics Named One of America’s Best Workplaces 2024 by Inc. Magazine

Company Earns Coveted Recognition by Inc. Magazine for Its Employee-First Culture and Supportive, Innovative Work Environment PASADENA, Calif., June 20, 2024 — Virtualitics, a leader

Maintenance Decision Intelligence AI Transforms Enterprise Asset Management

Virtualitics’ Maintenance Decision Intelligence AI Application Transforms Enterprise Asset Management

New AI-Powered Application Enhances Efficiency of the Scheduling Process by More Than 30% PASADENA, Calif., June 18, 2024 — Virtualitics, a leader in AI decision

Why Implementing Decision Intelligence Requires Data Transparency

For most business leaders, making decisions is no longer a simple process. Increased globalization, the rise of digital and hybrid workplaces, and the rapid development

Virtualitics named a Sample Vendor in 2024 Gartner Hype Cycle for Analytics and Business Intelligence.