level zero problem solving

Linux Tutorial

Bash Scripting Tutorial

HTML Tutorial

CSS Tutorial

Website Development Challenges

Binary Tutorial

Regular Expressions Tutorial

Boolean Algebra Tutorial

Solve the Cube

PyGame Tutorial

Problem Solving Skills

Basic Graphic Design

Programming Challenges

Software Design and Development

micro:bit tutorial

• Introduction
• Concept Triptychs

Fixing that which is broken and making everything else awesome!

Learn creative problem solving skills and techniques. Also a discussion on the process and theory behind powerful problem solving and creative thinking.

When you are problem solving you are doing one of two things. Either you are fixing something that is broken, or you are building something. If you are building something, it could be a completely new something, or it could be an improved something. Read on to discover how to do these effectively.

Some people are naturally good at problem solving. Others are not (yet). Most are somewhere in between. A lot of people underestimate just how good they are at problem solving. Many don't realise just how much problem solving they actually do. Problems are all around us and we solve them all the time, be it something as simple as deciding whether to put that extra blanket on the bed tonight, or something a little more difficult like constructing a plane out of old junk you've found in the garage.

Wherever you are at, you can improve your problem solving skills. With the right mindset, problem solving is not that difficult, and is quite fun. Fixing things and building things is always a rewarding task. The more you problem solve, the better you will get too.

The Right Attitude

To be an awesome problem solver, first we need to start with the right attitude.

It can be good to write some of these down in places you will see them regularly so you are always reminding yourself.

Be Open Minded

Be open to the possibility that:

• The actual problem you have may be different to the problem you think you have.
• What you think is causing the problem may not actually be what is causing the problem.
• The best option may be to throw out your solution and start again in a different direction.
• Your beliefs on anything and everything could be wrong.

Great new ideas often come from challenging our assumptions about how things do or should work. Always be open to different opinions. I'm not saying you have to agree with them but you should take the time to appreciate and understand them.

Be Inquisitive

Always be looking around and noticing things. Try and notice everything, not just what is (or what you think is) directly related to the problem area. Often, things which you don't suspect have anything to do with what you are looking at actually do.

• things that seem odd or stand out.
• any patterns.
• anything that is there that maybe shouldn't be.
• anything that is missing.

These are often good starting places to find clues.

A good way to develop inquisitiveness is to do a little activity when you are travelling (to or from work, school, etc). Look for things that stand out, or patterns, then ask yourself why they may have been done like that. eg. Why is that building 3 stories when all the others are two? Every third tree on this street is much wider than the others, why?

Ask the Right Questions

This is one of those things you get much better at with practice. Often when a student is stuck I find that I can get them unstuck by simply asking them the right questions.

I like to ask these general questions (about anything and everything):

• What is it actually doing?
• How is it actually doing it?
• How effectively is it performing its purpose?
• Why is it doing it/ what would happen if it was taken out?
• What is it's relationship with other components?
• What could I do to test that assumption?

You will also find that by asking these questions, new questions arise specific to the situation or problem. This is good. The more questions you can ask the better.

In answering these questions, always try and find proof or evidence. Try to rely as little as possible on assumptions.

It amazes me how often a student has asked for help in solving a problem and in the middle of explaining the problem to me they have realised the solution. I think it is largely because when you try to explain something to someone else you have to slow down. Speech is naturally a slower process than thought and you also tend to phrase it differently when you tell the problem to someone else rather than just thinking internally.

Slowing down can be difficult to do but can make a big difference.

Another approach to help slow things down is to come back to a problem. Leave it for a day or two (or an hour or two) and come back to it. You'll be surprised how much of a difference it can make.

Don't be Lazy

I see many students take the Where's Wally approach to problem solving. That is, they randomly, rapidly skim, hoping to find the solution. Effective problem solving is more like piecing together a jigsaw puzzle . We first look at all the pieces, trying to find the ones that stand out (such as corners and edges). Then we attack the problem from there and as the picture builds, the easier it gets. It's more work but the outcome is always much better.

Technology is great. Alas, it has also made us lazy. The result of search engines like Google and devices that increasingly hide the processing away from us is that we just expect the answers and results to be at our fingertips. Many people are just seeking that silver bullet so they don't actually have to do any work.

Effective problem solving doesn't work like that however. Don't be lazy. Read all the material, don't just skim it. Often the difference between something working and not is a little detail you have missed. You may get something that works by skimming but if you don't understand why it works then you won't be able to adapt it easily to other scenarios.

Don't Panic

Panic is the initial reaction to many when faced with a problem. The seasoned problem solver however is calm and relaxed in the face of problems as they know that solutions always exist and with the right approach (outlined below) they will triumph.

(The exception to this is velociraptors. If you find yourself looking into the steely cold eyes of one or more of these I don't like your chances. Unless, of course, Chuck Norris happens to be standing next to you.)

When you are calm you are able to think clearer and more creatively.

Make Sure You are Solving the Right Problem

When you start looking at a problem and it seems really simple, you don’t really understand the complexity of the problem. Then you get into the problem, and you see that it’s really complicated, and you come up with all these convoluted solutions. That’s sort of the middle, and that’s where most people stop… But the really great person will keep on going and find the key, the underlying principle of the problem — and come up with an elegant, really beautiful solution that works.

Sometimes the problem you initially see is not the problem at all. It is a symptom of the actual problem. If you solve the symptom you aren't solving the real problem, you're just putting a band-aid on it.

A great tool to help in identifying the Underlying Princple is Concept Triptychs .

Another good approach for this is to continually ask why until you can't really go any further without getting silly or adversely impacting other areas. eg:

My bike wheel keeps getting punctures. I initially thought the problem was that the innertube was dodgy.

Or another situation. This time it has been proposed that an intranet would improve the running of the company.

• The Process

Assess the Situation

So we have become a zen master and resisted the urge to panic. Awesome! Now we need to assess the situation. It will be one of the following:

• We are fixing a problem
• We are creating something new

If you are fixing a problem then you need to find as much information as possible to help you work out what has gone wrong and why. Read any error messages carefully. If there is any output, look over it to see if there are any useful clues.

If you are creating something then you should consider what resources you have available to build with. Consider all resources, even if can't see how they would be useful. Sometimes with a bit of creativity you can find great uses for seemingly unneeded resources.

Sometimes we may need to do some testing to help assess the situation.

• If I change the input do I get similar error messages or output?
• Are there other things I could do which will give me other information to compare the output against to better understand it?
• Can I break my solution down into several steps and verify that each step is behaving as intended?

Create a Hypothesis

Once we have looked at the situation we hopefully have some clues to help us understand it better. Now we need to make a guess at what we need to do to get closer to an acceptable solution.

There is a reasonable chance (especially to begin with) that your hypothesis will be incorrect. That is ok, we have to start somewhere. With experience you will get better at assessing the situation and working out what needs to be done however.

It is a good idea to start at the very bottom, with the most obvious culprits, and work your way up. eg. Maybe our internet is not working. First we may check another web page. Then we may check our network cable is plugged in. etc. and work our way up.

Test Your Hypothesis

Go ahead and put your idea into practice. Think also about how you may work out if it worked as intended or not. This will be similar to step 1 where we assessed the situation.

Rinse and Repeat

Once you've put your hypothesis into practice you will get a result. If the result is what you were after then great. If not then we need to go back to step 1 and assess the situation. We need to discover why our hypothesis was wrong then learn from that and create and test a new hypothesis. With each iteration we should discover a little more and that extra knowledge will help us get closer next time.

Even if we get a favorable result, sometimes it can be good to still have another go or two. Often our first successful attempt is a little bit messy and there is room for improvement. With a few more iterations we may get a cleaner solution and we'll also increase our knowledge. Both will help us with future problem solving.

• Function over Purpose

A common trait of those that are really good problem solvers is that they can find creative new ways to use existing items. So for instance, a hammer is not just for hitting nails. It could also be used as a doorstop, or a paperweight. It could be used as an instrument if you bang it against the right object. Maybe, if the end is the right size, it can be used to plug a hole. etc.

It is common for people to see items and only see them for their intended purpose. The reality is there is nothing stopping you, and it's perfectly fine, to use almost anything in almost any way you can imagine. What you need to do is stop thinking about items in terms of their purpose and start thinking about them in terms of their characteristics.

The technology we have around us today is especially flexible in how it can be used. Sometimes this can be hard to appreciate as the functioning of the software is hidden from us and the functions they perform are often abstract. We need to take the time to appreciate what the software is actually doing and how it does it. The naming of software can also sometimes work against us. For example on the Unix command line there is a command mv which is used to move a file. It's name implies that it's function is to only move files. When we understand how it works however we then come to realise that it can also be used to rename files, and in conjunction with other aspects of Unix, make files either hidden or not hidden.

Develop this thinking habit

A simple way to develop this habit of thinking is to play a game. This game can be played by yourself of with friends. All you need to do is pick a simple everyday item (it could be anything, a shoe or a pen for example) then try and come up with as many different uses as you can for it (beyond it's normal intended usage).

When playing with friends you can take it in turns and see who can come up with the most creative idea or who is the first person to not be able to come up with an idea.

Relationships Between Things

Related to the previous bit is the idea of relationships. What we are talking about here is how the actions of one thing affect that of other things. Being able to see these relationships is very important to problem solving.

• Fixing something is often the case of identifying the relationship between what is causing the problem and what is displaying the symptoms, then instigating actions to either reduce or counter those effects.
• Creating something is often the case of putting things together to create new relationships which achieve the desired outcome.

In both situations, discovering the relationships may not be easy. The right attitude , combined with the right process is very useful however.

• General Tips

Break the problem down

It is much easier to solve a series of small problems as opposed to one large problem. Often we try to just solve the problem in one go as we think this will be quicker and involve less work. This can come back to bite you however as you end up spending more time, trying to figure out where you went wrong, than you gained.

There are two ways I like to tackle this. Both will work nicely, it's just a matter of which you prefer or which suits the particular problem better.

• Break the problem in 1/2. Then break each of those problems in 1/2. Keep doing this until the problems are nice and manageable.
• Identify the easiest thing you can do which will get you closer to the solution and do that. Now identify the next easiest step and do that. Repeat until you have a fully working solution.

For both of these approaches you may need a few attempts before you get it right. For instance you may break the problem in 1/2 in a way that makes it more difficult as both sides rely upon parts of the other side. Don't be afraid to throw it out and have another go if it doesn't seem right.

Do the easiest step you can to get you closer to the solution. Then the next easiest. Repeat until done.

At least 3 solutions

Whenever you're about to do something of a reasonable level of complexity you should always try and consider at least 3 different ways of solving it. Those solutions should ideally be as diverse as possible.

Whenever we tackle a problem, our mind first tries to think of similar situations we have encountered in the past and work off those experiences. Often there are better solutions and by considering alternatives rather than just diving in with your first impression you'll discover a lot more.

The Uninterested Observer

Once you get really stuck into a problem your mind has a tendency to get focused. This can be bad as you easily miss things that should be obvious. It's very easy to get hung up on a particular line of thinking and completely miss alternatives.

Telling or showing another person your problem can be a great source of inspiration. Things that you have overlooked or decided not to consider will appear obvious to them. They are also not personally attached to the problem and so will see it from a different point of view to you.

Have you ever written an assignment and proof read it several times convinced it is perfect. Then a friend has glanced at it and instantly spotted a typo or mistake?

There is a lot to be said for thinking before you act. Sometimes however, the best way to start moving towards a solution is to dive in and start trying things. The first thing (or things) you try may not work out but they will help you understand the problem better, find out what works and what doesn't, and progress towards an ideal solution. Don't get stuck in analysis paralysis .

• Computing Specific Tips
• If you write some code, make sure you run it. Make sure you test it with different input to check if all the different branches operate as intended.
• If data is written into a file, make sure you look at the contents of the file and that those contents seem reasonable to you.
• The computer will do exactly what we tell it to do. The problem is that sometimes what we asked the computer to do and what we think we asked the computer to do are very different things.
• Just because you didn't get an error message doesn't necessarily mean it worked as intended. Make sure the output is what you expect it to be.
• Print variables out at different points to make sure they are behaving as intended?

Check for Typos or Silly Little Mistakes

They are easy to make and can be the difference between something working exactly as intended or spewing forth a screen full of error messages. If you feel confident that what you have entered is correct then spelling errors are one of the first places you should look. Check further up in your code too. Sometimes the computer is telling you there is an error on line 57 but the actual error is a missing semicolon ( ; ) or closing bracket ( } ) on line 24.

Writing clean indented code is one of the easiest ways to avoid these problems. Use a text editor that does syntax highlighting too as that also makes errors easy to detect.

Formatting can also be important when writing code or commands. Sometimes something as simple as the presence or absence of a space can break your solution entirely.

• Logic and Creativity Games

One of the best ways to improve these skills is through practice. The more problems you tackle and persevere with to find solutions, the better you will become. The more you learn the more resources you will also have at your disposal to use in different ways so working through the tutorials you'll find on this site is good for you too.

Another great way to enhance these skills is through games. Games are fun and you learn best when you're having fun. Below you will find some good games to play. There are many more out there though so these are just to get you started.

• Mastermind - Good for developing logical thinking.
• Set - Develop your powers of observation.
• Creativity Games - many fun and quick word games.
• 2048 - can you plan far enough ahead?
• Petals around the rose - another good observation puzzle.
• Ultimate Tic Tac Toe - again, good for planning ahead.
• Rubiks Cube

Keep it varied. The more different games you play the better.

If you have any suggestions for other games that could be listed here then let us know too.

• Learning Theory

I know we're talking about problem solving here but learning and problem solving actually overlap quite a bit. Solving a problem often involves learning more about the particular problem area. There's quite a bit we can talk about in terms of learning theory but I think one of the most important is the stages we go through in terms of our development.

Blooms Taxonomy is a way of distinguishing different levels in terms of a persons general ability or skill level. Here is my summarised version tailored to problem solving.

• Level 0 - I can identify what the actual problem is and what's causing it.
• Level 1 - I can find/ identify an existing solution (created by someone else) and am able to apply it.
• Level 2 - I can take a pre-made solution and tweak it to better suit this particular problem.
• Level 3 - I can create my own solutions and don't need the help of others.
• Level 4 - I can evaluate a solution and identify how good/ robust/ elegant it is.

You will probably progress through these levels as you get better at problem solving (particularly in a particular area of expertise). Level 0 is beginner and Level 4 is mastery.

People often get to Level 3 and think they have arrived, they are Grand Masters and all should look in awe upon their brilliance. Don't let this be you. Many catastrophes have been caused by people who were at Level 3 but really needed to be at Level 4.

A little knowledge can be a dangerous thing.

If you found this page useful, don't keep it a secret. Tell your friends about it. Share it on your favourite social media. Include a link to it on your blog. etc.

• Section Breakdown
• Solve the Right Problem
• Relationships

Education is the kindling of a flame, not the filling of a vessel. - Socrates

Contact | Disclaimer

Regular Expressions

Problem Solving

Basic Design Tutorial

micro:bit Tutorial

Spreadsheets Tips and Hints

Problems Solving Level One

Codewars 8 Kyu

Video Number 1

Video number 2.

• Simple Multiplication
• Century From Year
• Is N Divisible By X And Y?

Video Number 3

• Even Or Odd
• Reversed Strings
• Convert A Boolean To A String

Video Number 4

• Price Of Mangoes
• Quarter Of The Year
• Invert Values

Video Number 5

• Remove Exclamation Marks
• Find Maximum and Minimum Values of a List

Video Number 6

• Keep Hydrated!
• Beginner – Reduce But Grow
• Convert a String to a Number!

Video Number 7

• Remove First and Last Character
• String Repeat
• Remove String Spaces

Video Number 8

• All Star Code Challenge #18
• Beginner Series #2 Clock
• altERnaTIng cAsE ALTerNAtiNG CaSe

Video Number 9

• Will You Make It ?
• Is The String Uppercase ?
• Basic Mathematical Operations

Video Number 10

• Counting Sheep
• Square(n) Sum
• Count Odd Numbers Below N

Video Number 11

• What’s The Real Floor?
• Calculate Average
• Return Negative

Video Number 12

• Swap Values
• Drink About
• Is He Gonna Survive ?

Video Number 13

• Correct The Mistakes Of The Character Recognition Software
• Rock Paper Scissors!
• Jenny’s Secret Message

Video Number 14

• Get Character From ASCII Value
• Get Nth Even Number
• What Is Between ?

Video Number 15

• Abbreviate a Two Word Name
• MakeUpperCase
• You Only Need One – Beginner

Video Number 16

• Switch it Up!
• Opposites Attract
• Grasshopper – Terminal Game Move Function

Video Number 17

• Sum Of Differences In Array
• Grasshopper – Messi Goals Function
• Get Planet Name By ID

Video Number 18

• Array Plus Array
• If You Can’t Sleep, Just Count Sheep
• The Feast of Many Beasts

Video Number 19

• Count The Monkeys
• Sum The Strings
• Parse Nice Int From Char Problem

Video Number 20

• Multiplication Table For Number
• Do I Get A Bonus?
• Opposite Number

Video Number 21

• 101 Dalmatians – Squash The Bugs, Not The Dogs!
• Convert A Number To A String!

Video Number 22

• Sort And Star
• Reversed Sequence
• L1: Set Alarm

Video Number 23

• Sum Without Highest And Lowest Number
• Grasshopper – Terminal Game Combat Function
• Find The Smallest Integer In The Array

Video Number 24

• Sum Of Positive
• Volume Of A Cuboid
• Get The Mean Of An Array

Video Number 25

• Fake Binary
• The Falling Speed Of Petals
• Replace All Vowel To Exclamation Mark In The Sentence

Video Number 26

• Convert A String To An Array
• Is It Even?
• How Good Are You Really?

Video Number 27

• Twice As Old
• DNA To RNA Conversion
• Grasshopper – Check For Factor

Video Number 28

• Convert Number To Reversed Array Of Digits
• Is it A Palindrome?
• Reversed Words

Video Number 29

• Powers Of 2
• Grasshopper – Summation
• Thinkful – Logic Drills: Traffic Light

Video Number 30

• My Head Is At The Wrong End!
• Gravity Flip
• Collatz Conjecture (3n+1)

Video Number 31

• Count Of Positives / Sum Of Negatives
• Expressions Matter
• Pythagorean Triple

Video Number 32

• Total Amount Of Points
• Simple Fun #1: Seats in Theater

Video Number 33

• Grasshopper – Grade Book
• Beginner Series #1 School Paperwork

Video Number 34

• Transportation On Vacation
• Lario and Muigi Pipe Problem
• Student’s Final Grade

Video Number 35

• Area or Perimeter
• Find Numbers Which Are Divisible By Given Number
• Returning Strings

Made With Love By Osama Elzero © 2024

• Product overview
• All features
• Latest feature release
• App integrations

CAPABILITIES

• project icon Project management
• Project views
• Custom fields
• Status updates
• goal icon Goals and reporting
• Reporting dashboards
• workflow icon Workflows and automation
• portfolio icon Resource management
• Capacity planning
• Time tracking
• my-task icon Admin and security
• Admin console
• asana-intelligence icon Asana AI
• list icon Personal
• premium icon Starter
• briefcase icon Advanced
• Goal management
• Organizational planning
• Campaign management
• Creative production
• Content calendars
• Marketing strategic planning
• Resource planning
• Project intake
• Product launches
• Employee onboarding
• View all uses arrow-right icon
• Project plans
• Team goals & objectives
• Team continuity
• Meeting agenda
• View all templates arrow-right icon
• Work management resources Discover best practices, watch webinars, get insights
• Customer stories See how the world's best organizations drive work innovation with Asana
• Help Center Get lots of tips, tricks, and advice to get the most from Asana
• Asana Academy Sign up for interactive courses and webinars to learn Asana
• Developers Learn more about building apps on the Asana platform
• Community programs Connect with and learn from Asana customers around the world
• Events Find out about upcoming events near you
• Partners Learn more about our partner programs
• Asana for nonprofits Get more information on our nonprofit discount program, and apply.

Featured Reads

• Project management |
• What is 8D? A template for efficient pr ...

What is 8D? A template for efficient problem-solving

How you respond when problems arise is one of the most defining qualities of a manager. Luckily, there are tools you can use to master problem-solving. The 8D method of problem-solving combines teamwork and basic statistics to help you reach a logical solution and prevent new issues from arising.

You’ve spent months overseeing the development of your company's newest project. From initiation, planning, and execution, you’re confident this may be your best work yet.

Until the feedback starts rolling in.

There’s no sugar-coating it—things don’t always go as planned. But production or process issues are hardly a signal to throw in the towel. Instead, focus on honing your problem-solving skills to find a solution that keeps it from happening again. 

The 8D method of problem solving emphasizes the importance of teamwork to not only solve your process woes but prevent new ones from occurring. In this guide, we’ll break down what 8D is, how to use this methodology, and the benefits it can give to you and your team. Plus, get an 8D template to make solving your issue easier. 

What is 8D?

The eight disciplines (8D) method is a problem-solving approach that identifies, corrects, and eliminates recurring problems. By determining the root causes of a problem, managers can use this method to establish a permanent corrective action and prevent recurring issues. 

How do you use the 8D method?

The 8D method is a proven strategy for avoiding long-term damage from recurring problems. If you’re noticing issues in your workflow or processes, then it’s a good time to give this problem-solving method a try. 

To complete an 8D analysis, follow “the eight disciplines” to construct a statistical analysis of the problem and determine the best solution.

The eight disciplines of problem-solving

8D stands for the eight disciplines you will use to establish an 8D report. As you may notice, this outline starts with zero, which makes nine total disciplines. The “zero stage” was developed later as an initial planning stage. 

To illustrate these steps, imagine your organization experienced a decline in team innovation and productivity this past year. Your stakeholders have noticed and want to see changes implemented within the next six months. Below, we’ll use the 8D process to uncover a morale-boosting solution.

D0: Prepare and plan

Before starting the problem-solving process, evaluate the problem you want to solve. Understanding the background of the problem will help you identify the root cause in later steps. 

Collect information about how the problem has affected a process or product and what the most severe consequences may be. Planning can include:

Gathering data

Determining the prerequisites for solving the problem

Collecting feedback from others involved

If we look back at our example, you may want to figure out whether this decline in morale is organization-wide or only applies to a few departments. Consider interviewing a few employees from different departments and levels of management to gain some perspective. Next, determine what knowledge and skills you will need to solve this lapse in productivity. 

D1: Form your team

Create a cross-functional team made up of people who have knowledge of the various products and workflows involved. These team members should have the skills needed to solve the problem and put corrective actions in place. 

Steps in this discipline may include:

Appointing a team leader

Developing and implementing team guidelines

Determining team goals and priorities

Assigning individual roles

Arranging team-building activities

From our example, a solid team would consist of people with first-hand experience with the issues—like representatives from all departments and key people close to workshop-level work. You may also want to pull someone in from your HR department to help design and implement a solution. Most importantly, make sure the people you choose want to be involved and contribute to the solution.

D2: Identify the problem

You may have a good understanding of your problem by now, but this phase aims to break it down into clear and quantifiable terms by identifying the five W’s a and two H’s (5W2H):

Who first reported the problem?

What is the problem about?

When did it occur and how often?

Where did it occur (relating to the sector, supplier, machine, or production line involved)?

Why is solving the problem important?

How was the problem first detected?

How many parts/units/customers are affected?

Use your team’s insights to answer these questions. From our example, your team may conclude that: 

Employees feel overwhelmed with their current workload. 

There is no real structure or opportunity to share new ideas.

Managers have had no training for meetings or innovation settings.

Disgruntled employees know they can achieve more—and want to achieve more—even if they seem disengaged.

Once you answer these questions, record an official problem statement to describe the issue. If possible, include photos, videos, and diagrams to ensure all parties have a clear understanding of the problem. It may also help to create a flowchart of the process that includes various steps related to the problem description.

D3: Develop an interim containment plan

Much like we can expect speedy first aid after an accident, your team should take immediate actions to ensure you contain the problem—especially if the problem is related to customer safety. 

An interim containment plan will provide a temporary solution to isolate the problem from customers and clients while your team works to develop a permanent corrective action. This band-aid will help keep your customers informed and safe—and your reputation intact.

Because your findings revealed workers were overworked and managers lacked training, your team suggests scheduling a few mandatory training sessions for leaders of each department covering time and stress management and combating burnout . You may also want to have a presentation outlining the topics of this training to get key managers and stakeholders interested and primed for positive upcoming changes. 

D4: Verify root causes and escape points

Refer back to your findings and consult with your team about how the problem may have occurred. The root cause analysis involves mapping each potential root cause against the problem statement and its related test data. Make sure to test all potential causes—fuzzy brainstorming and sloppy analyses may cause you to overlook vital information. 

In our example, focus on the “why” portion of the 5W2H. You and your team identify six root causes:

Managers have never had any training

There is a lack of trust and psychological safety

Employees don’t understand the objectives and goals

Communication is poor

Time management is poor

Employees lack confidence

In addition to identifying the root causes, try to pinpoint where you first detected the problem in the process, and why it went unnoticed. This is called the escape point, and there may be more than one. 

D5: Choose permanent corrective actions

Work with your team to determine the most likely solution to remove the root cause of the problem and address the issues with the escape points. Quantitatively confirm that the selected permanent corrective action(s) (PCA) will resolve the problem for the customer. 

Steps to choosing a PCA may include:

Determining if you require further expertise

Ensuring the 5W2Hs are defined correctly

Carrying out a decision analysis and risk assessment

Considering alternative measures

Collecting evidence to prove the PCA will be effective

Your team decides to roll out the training used in the interim plan to all employees, with monthly company-wide workshops on improving well-being. You also plan to implement meetings, innovation sessions, and team-coaching training for managers. Lastly, you suggest adopting software to improve communication and collaboration. 

D6: Implement your corrective actions

Once all parties have agreed on a solution, the next step is to create an action plan to remove the root causes and escape points. Once the solution is in effect, you can remove your interim containment actions.

After seeing success with the training in the interim phase, your stakeholders approve all of your team’s proposed PCAs. Your representative from HR also plans to implement periodic employee wellness checks to track employee morale .

To ensure your corrective action was a success, monitor the results, customer, or employee feedback over a long period of time and take note of any negative effects. Setting up “controls” like employee wellness checks will help you validate whether your solution is working or more needs to be done. 

D7: Take preventive measures

One of the main benefits of using the 8D method is the improved ability to identify necessary systematic changes to prevent future issues from occurring. Look for ways to improve your management systems, operating methods, and procedures to not only eliminate your current problem, but stop similar problems from developing later on.

Based on our example, the training your team suggested is now adopted in the new manager onboarding curriculum. Every manager now has a “meeting system” that all meetings must be guided by, and workloads and projects are managed as a team within your new collaboration software . Innovation is improving, and morale is at an all-time high!

D8: Celebrate with your team

The 8D method of problem-solving is impossible to accomplish without dedicated team members and first-class collaboration. Once notes, lessons, research, and test data are documented and saved, congratulate your teammates on a job well done! Make an effort to recognize each individual for their contribution to uncovering a successful solution.

8D report template and example

Check out our 8D report template below to help you record your findings as you navigate through the eight disciplines of problem solving. This is a formal report that can be used as a means of communication within companies, which makes for transparent problem-solving that you can apply to the entire production or process chain.

Benefits of using the 8D method

The 8D method is one of the most popular problem-solving strategies for good reason. Its strength lies in teamwork and fact-based analyses to create a culture of continuous improvement —making it one of the most effective tools for quality managers. The benefits of using the 8D method include: 

Improved team-oriented problem-solving skills rather than relying on an individual to provide a solution

Increased familiarity with a problem-solving structure

A better understanding of how to use basic statistical tools for problem-solving

Open and honest communication in problem-solving discussions

Prevent future problems from occurring by identifying system weaknesses and solutions

Improved effectiveness and efficiency at problem-solving

Better collaboration = better problem solving

No matter how good a manager you are, production and process issues are inevitable. It’s how you solve them that separates the good from the great. The 8D method of problem solving allows you to not only solve the problem at hand but improve team collaboration, improve processes, and prevent future issues from arising. 

Try Asana’s project management tool to break communication barriers and keep your team on track.

Related resources

New site openings: How to reduce costs and delays

8 steps to write an effective project status report

Provider onboarding software: Simplify your hiring process

Inventory management software: How to control your stock

Proactive IT Support Strategies: Is Level 0 Support Right for You?

It’s a running gag in IT that end users tend to ask for help with the same few things: “My computer is slow,” “I can’t connect to the printer,” etc. The goal of Level 0 support is to lessen some of those pain points by resolving incidents before they reach the service desk through a combination of IT self-service and self-healing.

When end users contact IT for help, it’s not necessarily because they’re incapable of solving the problem themselves. Often, there’s simply no good process in place for them to do so.

We’re used to self-service in our everyday lives. Forgot your password? No problem. Click a link, open an email, and reset it yourself. This functionality is sometimes missing in the enterprise, particularly with custom-built applications.

Alternatively, self-healing decreases end-user involvement, but the intent is the same: deflect incidents away from Level 1. Self-healing is a form of IT automation where scripting is used to detect and resolve well-known issues without human intervention.

Through self-service and self-healing, Level 0 extends the “shift-left” approach within the support tiers model, which often includes Levels 1,2, and 3. By strategically moving incident resolution closer to the source, organizations can lower IT support costs, free up higher-level IT staff to focus on other projects, improve end-user experience, and help end users get back to work sooner.

Do You Need Level 0? Ask Yourself These Questions

Does your organization have…

• high ticket volumes for repeat incidents?
• high numbers of incidents that are generally easy to resolve, such as trouble connecting to peripherals?
• a global workforce that may not have access to live 24/7 IT support?
• users that require anytime, anywhere access to their technology?
• room for improvement in service quality and end-user satisfaction?

If your IT team deals with any of the above challenges, Level 0 support could help.

What Level 0 Support Looks Like

To illustrate how Level 0 would operate in an everyday IT support scenario, let’s look at different ways of approaching the same incident. Say a user is experiencing performance degradation and application crashes due to low disk space. Depending on the organization’s practices, here are several ways the issue could be resolved:

• The user attempts to solve the issue herself (without self-service resources). The user may have followed the system’s prompts to clear out old files but is still left with low space that quickly fills up as she continues working. She doesn’t know that her application issues are related to low disk space and contacts IT support.
• The user contacts IT support (Level 1) . The user calls or emails the service desk to file a ticket. A Level 1 technician files the ticket and it is placed in a queue. Assuming no higher-priority items are called in, a technician investigates the user’s system and finds the disk issue along with large video files in the user’s downloads folder. The technician instructs the user to remove these files if no longer needed.
• IT notices the issue and solves it proactively (Level 2) . A Level 2 technician receives a notification that the user is experiencing a high number of application crashes. The technician investigates the user’s system and discovers that low disk space is the root cause of the issue. They contact the user and suggest deleting large files or moving them to a drive with more free space.
• If she locates an answer, the user checks for drives with more free space, deletes old downloads, and empties her trash.
• If she cannot locate an answer, or if the fix she found doesn’t work, the user contacts IT.
• IT has an automated disk clean up script in place (Level 0). Knowing that the organization has provisioned small drives, IT creates a script to run whenever free disk space falls below 10%. The user receives a notification asking if they would like to clear up space by removing downloads older than 14 days.

Even in this simple example, there are many possible paths for resolving the same incident. What path resolution will take depends on whether the organization supports Level 0, the quality of self-service resources, a user’s technical knowledge, and IT’s level of visibility into the issue.

Creating a Self-Service Culture

Organizations that already provide FAQ pages and/or knowledge base articles for end users will know that self-service can be tricky to get right. Self-service is only as valuable as it is usable, which means that any resources provided must satisfy several criteria:

• A high probable success rate (successful resolutions encourage users to continue to perform self-service)
• Easy to use (intuitive UX design; uses end-user-friendly terms)
• Easily accessible (resources are housed in one central location; good search functionality)

Additionally, users must be enabled and encouraged to use self-service resources through marketing and training. IT staff also must continue to manage and update knowledge base articles to ensure their relevancy.

Ultimately, the goal of self-service is not to push the responsibility of resolving incidents onto the end user. Rather, self-service should ideally improve end-user experience and productivity by helping end users solve problems more quickly and painlessly than a traditional IT interaction.

Self-Healing: To Automate or Not to Automate?

The issue with automation in IT has not been lack of interest so much as a shortage of viable solutions. One of the main setbacks to implementing self-healing is insufficient visibility into the current state of users’ systems and the issues they’re experiencing. Tickets and one-on-one interactions with end users only reveal so much. To understand potential use cases for automation in a service context, you need data on system usage and performance.

Consider the earlier example of a self-healing script that clears up disk space. To execute that script, you need 1) knowledge of whether low disk space is a common issue, 2) a mechanism to detect when disk space falls below a pre-defined threshold, and 3) a way to tie the script to the low disk space alarm so that it triggers automatically. These steps all require timely, detailed endpoint data.

Other examples of automation include starting/stopping services and restarting a system. In all cases, it’s imperative to consider how the script will affect the end user. For instance, it could be necessary to know whether the user is online, whether they are actively using an application, and what applications are most important to them (i.e. most actively used) to ensure that automation is not prioritized over end-user experience.

Some Benefits of Level 0

A well-executed Level 0 support practice brings the following benefits:

• Cost reduction: Level 0 helps optimize the support cycle by better matching incidents to the appropriate support tier. Incidents are resolved faster and at lower tiers, decreasing the cost associated with resolving the incident and lowering costs associated with downtime.
• Better quality of service: By reducing Level 1 incident volume, Level 0 makes it possible for technicians to tackle more critical issues as they arise, decreasing time-to-resolution. Self-healing also enables issues to be addressed proactively before an end user notices a problem.
• Improved end-user experience: Level 0 incorporates popular aspects of the consumer experience into enterprise IT. Robust self-service resources enable end users to more seamlessly resolve common issues.
• Enhanced service desk operations: Shifting left to Level 0 helps each support tier focus on work that will provide the most value to the organization. For Level 2 and Level 3, deflecting support requests means more time for higher-value IT projects and core engineering work.

Why Digital Experience Monitoring Is Essential

As with many pervasive IT challenges, IT support has struggled with 1) a lack of visibility into what is causing problems and 2) a lack of the quality data needed to resolve them. Without adequate data, IT’s only path for resolving issues is to involve the end user, sparking a back-and-forth that eats into productivity on both sides.

Here are some ways digital experience monitoring (DEM) helps with Level 0:

• Gathering data to inform the creation of KB articles and self-healing scripts. DEM provides insight into system and application performance as well as usage and end-user experience.
• Identifying common problems. DEM tools surface the incidents happening most frequently as well as ones that affect large amounts of users. They can also be used to quantify the impact of various issues in terms of lost productivity.
• Understanding the diverse needs of different user populations. DEM helps you segment users by persona, enabling you to better target candidates for self-healing.
• Integrating with ITSM tools. DEM tools can be used alongside ITSM solutions to integrate seamlessly into the support process, including providing more context for Level 1 support to potentially stop incidents from escalating as well as matching issues to the appropriate support level faster.
• Judging the success of the Level 0 implementation. DEM shows whether Level 0 is positively impacting performance and end-user experience.

Now that you’re a Level 0 pro, it’s time to evaluate how your organization could benefit from IT self-service and self-healing. Sign up for a free SysTrack demo to see how digital experience monitoring can help you achieve the benefits of a Level 0 approach.

Request a SysTrack demo

You may also be interested in…

• Three IT Self-Service Myths to Avoid
• What Is AIOps? Introducing SysTrack 8.4
• How to Increase Productivity with a Self-Healing Help Desk
• Proactive vs Reactive IT Support

Subscribe to the Lakeside Newsletter

Receive platform tips, release updates, news and more

Related Posts

“Shifting Left”: The Business Value of Moving IT Incident Resolution from L3 to L1 Technicians 

Lakeside vs. the Competition: A Closer Look at Why Lakeside SysTrack Cannot Cause a BSOD Outage

Digital Employee Experience: The Olympic Sport We Never Knew We Needed

Enroll in our Masterclass before August 31 and enjoy complementary lifetime membership at Corporate Rebels (10 spots left)

5 Levels Of Problem Solving: A Framework For (First-Time) Managers

Leading people can be tough. Taking the reins for the first time? It can feel like navigating a maze blindfolded. Many first-time managers feel that pit in their stomach. Without the right guidance or training, they often turn, unintentionally, into micromanagers. Nobody's dream scenario. But, good news - help's at hand!

A few weeks ago, I interviewed two workplace pioneers for one of our monthly live events (part of the Corporate Rebels Academy ). In this extremely insightful conversation, I had the pleasure of interviewing Edwin Jansen and Luz Iglesias. They both work at Raise (formerly known as Ian Martin Group).

The company employs over 450 people and has offices in the US, Canada, India, Ghana, and the Philippines. It is a certified B Corp and has been reinventing its management practices for about 8 years.

Its business? Recruitment.

Avoid becoming the all-mighty fixer

In 2017, Edwin and Luz took the lead on changing the company's way of working. They started in one of the company's subsidiaries and used it as a test ground for finding a better way to work.

When I asked Edwin, who was the leader of the subsidiary back then, what their biggest challenge was, his answer was clear:

"It was me. I had to change."

Edwin: "At the time, Luz pointed out a significant flaw in my approach: I had become a 'fixer'. To be clear, this wasn't a compliment.

"It highlighted a dangerous trajectory where I had become the go-to person for every problem, failing to empower my team members to think critically and solve issues independently.

"Doing so, I robbed people of the opportunity to learn and grow."

In a self-managing organization, leaders have to let go and give others the opportunity to step up. Edwin was very honest about his shortcomings:

"I simply didn't know how to do that. I had to let people find their own solutions, support them to take more ownership, and stop solving their problems for them."

Edwin and Luz decided to change.

5 levels of problem solving

Along the way, Edwin encountered a framework that helped him become a better leader.

The framework details 5 levels of problem solving - moving from highly dependent to highly independent. The clarity of the various 'levels' of problem solving helped him and his colleagues to create awareness on how they were operating, while also providing clear opportunities for growth (for all involved).

Here's how Edwin explained it:

To summarize, here are the 5 levels Edwin talked about:

• Level 1: The individual doesn't recognize the problem and doesn't know how to solve it.
• Level 2: The individual can identify the problem but doesn't know the solution.
• Level 3: The individual recognizes the problem and has considered multiple solutions, but is unsure which to choose.
• Level 4: The individual identifies the problem, has multiple solutions, and proposes one.
• Level 5: The individual has already encountered a problem, found a solution, and acted on it, and now reports the resolution post-action.

Wanna improve? Here's Edwin's advice:

"At any point, if you're a manager and someone comes to you, whatever level they come to you at, ask them to go one level up.

"And if you're not a manager and you're coming with problems, make sure you're at the highest level that you possibly can be."

Solid (and practical) advice.

Start decentralizing decision-making now

Decision-making is an art, but with the right framework, it becomes a systematic process that fosters growth and innovation.

The '5 levels of problem solving' is one of those frameworks that has the power to change the way you work immediately. Print it out, share it with your team, and follow Edwin's advice.

Eager for more tips, tools, and frameworks to improve your decision-making? We've got you covered.

With an in-depth course, lots of pioneering practices, and powerful tools, our Academy has everything you need to radically reinvent the way you work.

Start now. Click here .

Download: New Ways Of Working Guide

Unlock our in-depth guide spotlighting trends, tools, and best practices from over 150 pioneering organizations.

No spam. Unsubscribe anytime.

Download: Free Guide

Unlock our in-depth guide on trends, tools, and best practices from over 150 pioneering organizations.

Subscribe below and receive it directly in your inbox.

We respect your privacy. Unsubscribe at any time.

• See all features

Outbound Call Center

Inbound Contact Center

Business Phone System

IVR (Call Menu)

Call Forwarding

Sales Dialer

Call Recording

Advanced Analytics

Automated Call Distribution

Country Coverage

Is JustCall the right fit for you? Get hands-on with our interactive demo.

JustCall Email New

Work together on customer emails with your team

SMS & MMS Inbox

SMS Workflows

WhatsApp Inbox

SMS Copilot

SMS Campaigns

Customer stories

Avalon Pharmacy decreased call queries by 40% and saved 2 hours per day with JustCall

The only AI solution thats built for you

Conversation Intelligence

AI Coaching

Agent Assist

Automatic Call Scoring

Agent Analytics

Call Transcription

Moment Analysis

Sentiment Analysis

Master Your Minutes with AI-Driven Call Summaries & Transcriptions

CRM Integration

Call Disposition

Automated Call Scoring

Predictive Dialer

With JustCall, Pinnacle Career Institute sees its enrollment increase more than 52%

Integrate JustCall With The Tools You Love & Use

Platform Overview

The only platform you need for customer communication

Collaborate

Everyone from Meta Workspace is here! Check out JustCall Workspace.

IT & Operations

Customer Success

Customer Support

Recruitment

Real Estate

Financial Services

Fitness & Wellness

CrowdStreet reduces client response times by 100%

Glossary / Level Zero Solvable

What Is Level Zero Solvable?

Level zero solvable refers to customer support issues that can be resolved without the need for human intervention. These issues are typically addressed through automated systems, self-service options, or pre-defined solutions available in a knowledge base.

Level Zero Solvable: A Sneak Peek

Level zero solvable issues are designed to be handled by automated tools or self-service resources, allowing customers to find solutions quickly and efficiently. This approach helps reduce the workload on support teams by filtering out simple, repetitive queries that do not require human assistance. By providing customers with the means to resolve their own issues, businesses can improve response times and overall customer satisfaction.

These issues are often addressed through FAQs, chatbots, IVR systems, or online help centers. The goal is to empower customers to solve their problems independently, freeing up support agents to focus on more complex and high-priority cases. Implementing effective level zero solvable solutions can lead to significant cost savings and operational efficiencies for businesses.

Examples of Level Zero Solvable

A customer needs to reset their password and uses an automated password reset tool available on the company’s website.

A user encounters a common error message and finds a step-by-step troubleshooting guide in the online help center to resolve the issue.

A customer has a question about billing and uses a chatbot to get instant answers without waiting for a live agent.

Learn more about JustCall solutions

Improve your sales outreach and streamline prospect connections. Maximize productivity and achieve positive results in every call.

Reach customers across 70+ countries with text messages. Connect with customers in real-time.

Experience the power of AI in customer conversations. Coach agents in real-time and turn each conversation into a delightful experience.

You are here

Zero based problem solving, start fresh.

We tend to start a problem solving process with previous bias. Admitting that we can’t be totally free of bias, it is of great importance to start solving a new problem from as much neutral a standpoint as possible.

It will enable you to properly analyze the new problem on its own merit. After this analysis, if need be, you will use the known problem solving approaches, principles, techniques and tools suitable for solving this specific problem.

At the very beginning, before analyzing the problem well, you should have no opinion regarding how you will go about solving this problem.

You will start with a clean slate, a zero base and an alert and bias-free analytical mind, prepared for anything. As you proceed towards the solution step by step, gradually you will start forming rules and patterns that should be effective in solving this specific problem.

This is what we call the Principle of Zero Based Problem Solving.

Here we have used the abstract idea of zero based approach in problem solving to form a useful principle. The important concept here is the idea of zero based approach. It simply means, start fresh.

It is a powerful approach and can be used in many varied situations in our lives when we are doing tough tasks one after the other, specially under time pressure.

Examples are all around us. In many competitive tests, after two hard questions an easy question pops up, and the student might falter trying to solve the easy one in a complicated way. In competitive sports, professional players must focus on the present match fully, not influenced by any of the results of the previous matches. A doctor must analyze a patient's symptoms on its own merit. His previous patient's symptoms and analysis should no way affect his present analysis except as already formed useful rules.

And life is - solving a series of problems. Every such problem needs to be dealt with independently on its own merit.

Let's experience a problem solving process.

Learning from solving Four Square Problem

Problem statement:.

Question 1: Divide the yellow unshaded area in square A into two equal parts. Think over, it's easy.

Hoping that you have solved the first question let's face the second one.

Question 2: Divide the yellow unshaded area in square B into three equal parts. Look at the unshaded portion of square B and soon you should find the answer.

Assuming again that you have got the correct answer at this second step, let's face the third question now.

Question 3: Divide the yellow unshaded area in square C into four equal parts.

This may be a bit more difficult, but don't stop trying. Solve this question and measure your time.

Here comes the fourth and the last question. Be prepared.

Question 4: Divide the yellow area in square D into seven equal parts.

Think through, try seriously and then only go further.

Analysis and solution

Solution to question 1

Solution to the first question is easy. The red diagonal line divides the unshaded area in square A into two equal parts. It is generally visible to most people in a few seconds.

Solution to question 2

While looking for the second answer, we may visualize the empty area as a collection of three equal squares. Basically in this case, square B is visualized in terms of a square made up of four smaller squares. For this answer most probably you didn’t have to wait long. By visual inspection, the answer comes easily.

Solution to question 3

Perhaps the third question gave you some trouble. At least it gave us a little trouble. By previous bias we tried to think square C as a square made up of four squares. Within a few seconds we found our mistake - this approach won't take us to the solution. Basically the unshaded portion here consists of three equal squares. We need to divide it into four equal parts. Divide three into four - how?

How can we divide a three unit area into four equal parts?

Can we break down each part of the three part piece into four still smaller parts? If we do that we will get 3x4, that is, 12 smaller parts. These parts are equal squares. Oh yes, we have got the solution now. Combine three such adjacent squares together. Do it suitably three times and you will get your four equal areas. When you do the third time, the fourth area will automatically be created. The breakdown of the three small squares is shown below. The red dotted lines show the boundary of the squares.

Instead of dividing each of the three smaller squares of yellow unshaded area of square C into four equal still smaller squares, we could have divided each into four triangles. Grouping the resulting 12 triangles into four adjacent clusters of three triangles each would also have lead us to the desired solution. This is a second alternate solution .

While learning problem solving, we always ask, "In how many more ways this problem can be solved?" Advantages of this approach are threefold. This approach,

• strengthens your ability to see many paths to the solution that are not easily visible
• gives you a chance to evaluate the alternative paths to the solution, and
• strengthens your exhaustive thinking ability

We call this important approach as, Problem solving in many ways approach. As such it is not a problem solving resource - rather it is a problem solving skill building method. It simply states,

Solve an already solved problem in as many efficient ways as possible.

Being problem solvers we cannot forget the efficiency of a solution.

Coming back to our problem, which principles did we use till now? In the question 2, we straightway divided the empty space into three equal parts. That method we can’t follow here we have tried and found out. This learning was through analysis and application of Principle of Fresh Start or zero based problem solving.

This step of problem analysis is vital in learning the nature and anatomy of the problem. At this step we have also discovered that we need to divide a three unit area into four equal smaller pieces.

With this vital knowledge we further discovered that dividing each of the smaller squares into 4 still smaller squares (or triangles) and then grouping the resulting 12 squares (or triangles) into four groups would give us our solution.

We have used two inventive principles here:

First we have used: Principle of Segmentation (or dividing or fragmenting), and

Next we have used: Principle of Merging (or combining similar things).

These are two of the leading principles in the set of 40 inventive principles of TRIZ. These principles have very wide applications.

Solution to question 4

Let us come to the fourth solution now. In this case we don't have the unshaded area - the whole of square D is to be divided.

After the intial delay of about 10 seconds, we have straightway measured one side of the square D and divided the side into seven parts by drawing six straight lines parallel to the vertical side.

What is your experience in answering the fourth question?  Either you know the answer instantly or you find it difficult. If you get the solution instantly, you have used your intuition. Intuition is also a powerful mechanism towards problem solving, no second thoughts about it. Intuitive problem solving is fastest.

In emergent situations such as fire-fighting, front-line battle in a war split second decision making is the necessity and that kind of decision making is mostly intuitive based on experience and training.

Unfortunately for us, we were stalled again at this fourth step for about 10 seconds trying to segment the totally empty square D using the complex methods we had used in question 3 just before.

Previous bias

That was bias. This is one example of the need to start fresh without any bias. The more you move along the earlier path, further away you will move from the solution.

Quickly though we recognized our mistake and started to think fresh from a zero base. It became immediately apparent that we can divide the fully empty square into any number of equal pieces by just dividing one of its sides into equal portions and drawing straight lines parallel to its perpendicular side through these equal portions.

This is another kind of segmentation, but used differently. We are segmenting a space by segmenting a length.

This is more versatile, as by applying this concept you can actually divide the square into any number of equal portions. So we say,

Application of this concept here is more general in scope.

While solving the apparently easy four stage problem, we were stalled unexpectedly twice. At stage 3, the delay was minor, but the deliberately created misdirection at the transition from question 3 to question 4 caught us napping. That is where we became aware of the need of zero based problem solving without any previously acquired bias.

This is an academic artificially created problem, but we also realized,

While going through life and facing a stream of problems, misdirection from bias may happen anytime,

unless we always face a problem with a no-bias totally neutral mind.

  Next read

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Perspective
• Published: 25 January 2022

Intelligent problem-solving as integrated hierarchical reinforcement learning

• Manfred Eppe   ORCID: orcid.org/0000-0002-5473-3221 1   nAff4 ,
• Christian Gumbsch   ORCID: orcid.org/0000-0003-2741-6551 2 , 3 ,
• Matthias Kerzel 1 ,
• Phuong D. H. Nguyen 1 ,
• Martin V. Butz   ORCID: orcid.org/0000-0002-8120-8537 2 &
• Stefan Wermter 1  

Nature Machine Intelligence volume  4 ,  pages 11–20 ( 2022 ) Cite this article

5508 Accesses

34 Citations

7 Altmetric

Metrics details

• Cognitive control
• Computational models
• Computer science
• Learning algorithms
• Problem solving

According to cognitive psychology and related disciplines, the development of complex problem-solving behaviour in biological agents depends on hierarchical cognitive mechanisms. Hierarchical reinforcement learning is a promising computational approach that may eventually yield comparable problem-solving behaviour in artificial agents and robots. However, so far, the problem-solving abilities of many human and non-human animals are clearly superior to those of artificial systems. Here we propose steps to integrate biologically inspired hierarchical mechanisms to enable advanced problem-solving skills in artificial agents. We first review the literature in cognitive psychology to highlight the importance of compositional abstraction and predictive processing. Then we relate the gained insights with contemporary hierarchical reinforcement learning methods. Interestingly, our results suggest that all identified cognitive mechanisms have been implemented individually in isolated computational architectures, raising the question of why there exists no single unifying architecture that integrates them. As our final contribution, we address this question by providing an integrative perspective on the computational challenges to develop such a unifying architecture. We expect our results to guide the development of more sophisticated cognitively inspired hierarchical machine learning architectures.

This is a preview of subscription content, access via your institution

Access options

Access Nature and 54 other Nature Portfolio journals

Get Nature+, our best-value online-access subscription

24,99 € / 30 days

cancel any time

Subscribe to this journal

Receive 12 digital issues and online access to articles

111,21 € per year

only 9,27 € per issue

Buy this article

• Purchase on SpringerLink
• Instant access to full article PDF

Prices may be subject to local taxes which are calculated during checkout

Similar content being viewed by others

Phy-Q as a measure for physical reasoning intelligence

Hierarchical motor control in mammals and machines

Hierarchical generative modelling for autonomous robots

Gruber, R. et al. New Caledonian crows use mental representations to solve metatool problems. Curr. Biol. 29 , 686–692 (2019).

Article   Google Scholar  

Butz, M. V. & Kutter, E. F. How the Mind Comes into Being (Oxford Univ. Press, 2017).

Perkins, D. N. & Salomon, G. in International Encyclopedia of Education (eds. Husen T. & Postelwhite T. N.) 6452–6457 (Pergamon Press, 1992).

Botvinick, M. M., Niv, Y. & Barto, A. C. Hierarchically organized behavior and its neural foundations: a reinforcement learning perspective. Cognition 113 , 262–280 (2009).

Tomov, M. S., Yagati, S., Kumar, A., Yang, W. & Gershman, S. J. Discovery of hierarchical representations for efficient planning. PLoS Comput. Biol. 16 , e1007594 (2020).

Arulkumaran, K., Deisenroth, M. P., Brundage, M. & Bharath, A. A. Deep reinforcement learning: a brief survey. IEEE Signal Process. Mag. 34 , 26–38 (2017).

Li, Y. Deep reinforcement learning: an overview. Preprint at https://arxiv.org/abs/1701.07274 (2018).

Sutton, R. S. & Barto, A. G. Reinforcement Learning : An Introduction 2nd edn (MIT Press, 2018).

Neftci, E. O. & Averbeck, B. B. Reinforcement learning in artificial and biological systems. Nat. Mach. Intell. 1 , 133–143 (2019).

Eppe, M., Nguyen, P. D. H. & Wermter, S. From semantics to execution: integrating action planning with reinforcement learning for robotic causal problem-solving. Front. Robot. AI 6 , 123 (2019).

Oh, J., Singh, S., Lee, H. & Kohli, P. Zero-shot task generalization with multi-task deep reinforcement learning. In Proc. 34th International Conference on Machine Learning ( ICML ) (eds. Precup, D. & Teh, Y. W.) 2661–2670 (PMLR, 2017).

Sohn, S., Oh, J. & Lee, H. Hierarchical reinforcement learning for zero-shot generalization with subtask dependencies. In Proc. 32nd International Conference on Neural Information Processing Systems ( NeurIPS ) (eds Bengio S. et al.) Vol. 31, 7156–7166 (ACM, 2018).

Hegarty, M. Mechanical reasoning by mental simulation. Trends Cogn. Sci. 8 , 280–285 (2004).

Klauer, K. J. Teaching for analogical transfer as a means of improving problem-solving, thinking and learning. Instruct. Sci. 18 , 179–192 (1989).

Duncker, K. & Lees, L. S. On problem-solving. Psychol. Monographs 58, No.5 (whole No. 270), 85–101 https://doi.org/10.1037/h0093599 (1945).

Dayan, P. Goal-directed control and its antipodes. Neural Netw. 22 , 213–219 (2009).

Dolan, R. J. & Dayan, P. Goals and habits in the brain. Neuron 80 , 312–325 (2013).

O’Doherty, J. P., Cockburn, J. & Pauli, W. M. Learning, reward, and decision making. Annu. Rev. Psychol. 68 , 73–100 (2017).

Tolman, E. C. & Honzik, C. H. Introduction and removal of reward, and maze performance in rats. Univ. California Publ. Psychol. 4 , 257–275 (1930).

Google Scholar  

Butz, M. V. & Hoffmann, J. Anticipations control behavior: animal behavior in an anticipatory learning classifier system. Adaptive Behav. 10 , 75–96 (2002).

Miller, G. A., Galanter, E. & Pribram, K. H. Plans and the Structure of Behavior (Holt, Rinehart & Winston, 1960).

Botvinick, M. & Weinstein, A. Model-based hierarchical reinforcement learning and human action control. Philos. Trans. R. Soc. B Biol. Sci. 369 , 20130480 (2014).

Wiener, J. M. & Mallot, H. A. ’Fine-to-coarse’ route planning and navigation in regionalized environments. Spatial Cogn. Comput. 3 , 331–358 (2003).

Stock, A. & Stock, C. A short history of ideo-motor action. Psychol. Res. 68 , 176–188 (2004).

Hommel, B., Müsseler, J., Aschersleben, G. & Prinz, W. The theory of event coding (TEC): a framework for perception and action planning. Behav. Brain Sci. 24 , 849–878 (2001).

Hoffmann, J. in Anticipatory Behavior in Adaptive Learning Systems : Foundations , Theories and Systems (eds Butz, M. V. et al.) 44–65 (Springer, 2003).

Kunde, W., Elsner, K. & Kiesel, A. No anticipation-no action: the role of anticipation in action and perception. Cogn. Process. 8 , 71–78 (2007).

Barsalou, L. W. Grounded cognition. Annu. Rev. Psychol. 59 , 617–645 (2008).

Butz, M. V. Toward a unified sub-symbolic computational theory of cognition. Front. Psychol. 7 , 925 (2016).

Pulvermüller, F. Brain embodiment of syntax and grammar: discrete combinatorial mechanisms spelt out in neuronal circuits. Brain Lang. 112 , 167–179 (2010).

Sutton, R. S., Precup, D. & Singh, S. Between MDPs and semi-MDPs: a framework for temporal abstraction in reinforcement learning. Artif. Intell. 112 , 181–211 (1999).

Article   MathSciNet   MATH   Google Scholar  

Flash, T. & Hochner, B. Motor primitives in vertebrates and invertebrates. Curr. Opin. Neurobiol. 15 , 660–666 (2005).

Schaal, S. in Adaptive Motion of Animals and Machines (eds. Kimura, H. et al.) 261–280 (Springer, 2006).

Feldman, J., Dodge, E. & Bryant, J. in The Oxford Handbook of Linguistic Analysis (eds Heine, B. & Narrog, H.) 111–138 (Oxford Univ. Press, 2009).

Fodor, J. A. Language, thought and compositionality. Mind Lang. 16 , 1–15 (2001).

Frankland, S. M. & Greene, J. D. Concepts and compositionality: in search of the brain’s language of thought. Annu. Rev. Psychol. 71 , 273–303 (2020).

Hummel, J. E. Getting symbols out of a neural architecture. Connection Sci. 23 , 109–118 (2011).

Haynes, J. D., Wisniewski, D., Gorgen, K., Momennejad, I. & Reverberi, C. FMRI decoding of intentions: compositionality, hierarchy and prospective memory. In Proc. 3rd International Winter Conference on Brain-Computer Interface ( BCI ), 1-3 (IEEE, 2015).

Gärdenfors, P. The Geometry of Meaning : Semantics Based on Conceptual Spaces (MIT Press, 2014).

Book   MATH   Google Scholar  

Lakoff, G. & Johnson, M. Philosophy in the Flesh (Basic Books, 1999).

Eppe, M. et al. A computational framework for concept blending. Artif. Intell. 256 , 105–129 (2018).

Turner, M. The Origin of Ideas (Oxford Univ. Press, 2014).

Deci, E. L. & Ryan, R. M. Self-determination theory and the facilitation of intrinsic motivation. Am. Psychol. 55 , 68–78 (2000).

Friston, K. et al. Active inference and epistemic value. Cogn. Neurosci. 6 , 187–214 (2015).

Berlyne, D. E. Curiosity and exploration. Science 153 , 25–33 (1966).

Loewenstein, G. The psychology of curiosity: a review and reinterpretation. Psychol. Bull. 116 , 75–98 (1994).

Oudeyer, P.-Y., Kaplan, F. & Hafner, V. V. Intrinsic motivation systems for autonomous mental development. In IEEE Transactions on Evolutionary Computation (eds. Coello, C. A. C. et al.) Vol. 11, 265–286 (IEEE, 2007).

Pisula, W. Play and exploration in animals—a comparative analysis. Polish Psychol. Bull. 39 , 104–107 (2008).

Jeannerod, M. Mental imagery in the motor context. Neuropsychologia 33 , 1419–1432 (1995).

Kahnemann, D. & Tversky, A. in Judgement under Uncertainty : Heuristics and Biases (eds Kahneman, D. et al.) Ch. 14, 201–208 (Cambridge Univ. Press, 1982).

Wells, G. L. & Gavanski, I. Mental simulation of causality. J. Personal. Social Psychol. 56 , 161–169 (1989).

Taylor, S. E., Pham, L. B., Rivkin, I. D. & Armor, D. A. Harnessing the imagination: mental simulation, self-regulation and coping. Am. Psychol. 53 , 429–439 (1998).

Kaplan, F. & Oudeyer, P.-Y. in Embodied Artificial Intelligence , Lecture Notes in Computer Science Vol. 3139 (eds Iida, F. et al.) 259–270 (Springer, 2004).

Schmidhuber, J. Formal theory of creativity, fun, and intrinsic motivation. IEEE Trans. Auton. Mental Dev. 2 , 230–247 (2010).

Friston, K., Mattout, J. & Kilner, J. Action understanding and active inference. Biol. Cybern. 104 , 137–160 (2011).

Oudeyer, P.-Y. Computational theories of curiosity-driven learning. In The New Science of Curiosity (ed. Goren Gordon), 43-72 (Nova Science Publishers, 2018); https://arxiv.org/abs/1802.10546

Colombo, M. & Wright, C. First principles in the life sciences: the free-energy principle, organicism and mechanism. Synthese 198 , 3463–3488 (2021).

Article   MathSciNet   Google Scholar  

Huang, Y. & Rao, R. P. Predictive coding. WIREs Cogn. Sci. 2 , 580–593 (2011).

Friston, K. The free-energy principle: a unified brain theory? Nat. Rev. Neurosci. 11 , 127–138 (2010).

Knill, D. C. & Pouget, A. The Bayesian brain: the role of uncertainty in neural coding and computation. Trends Neurosci. 27 , 712–719 (2004).

Clark, A. Whatever next? Predictive brains, situated agents, and the future of cognitive science. Behav. Brain Sci. 36 , 181–204 (2013).

Clark, A. Surfing Uncertainty : Prediction , Action and the Embodied Mind (Oxford Univ. Press, 2016).

Zacks, J. M., Speer, N. K., Swallow, K. M., Braver, T. S. & Reyonolds, J. R. Event perception: a mind/brain perspective. Psychol. Bull. 133 , 273–293 (2007).

Eysenbach, B., Ibarz, J., Gupta, A. & Levine, S. Diversity is all you need: learning skills without a reward function. In International Conference on Learning Representations (ICLR, 2019).

Frans, K., Ho, J., Chen, X., Abbeel, P. & Schulman, J. Meta learning shared hierarchies. In Proc. International Conference on Learning Representations https://openreview.net/pdf?id=SyX0IeWAW (ICLR, 2018).

Heess, N. et al. Learning and transfer of modulated locomotor controllers. Preprint at https://arxiv.org/abs/1610.05182 (2016).

Jiang, Y., Gu, S., Murphy, K. & Finn, C. Language as an abstraction for hierarchical deep reinforcement learning. In Neural Information Processing Systems ( NeurIPS ) (eds. Wallach, H. et al.) 9414–9426 (ACM, 2019).

Li, A. C., Florensa, C., Clavera, I. & Abbeel, P. Sub-policy adaptation for hierarchical reinforcement learning. In Proc. International Conference on Learning Representations https://openreview.net/forum?id=ByeWogStDS (ICLR, 2020).

Qureshi, A. H. et al. Composing task-agnostic policies with deep reinforcement learning. In Proc. International Conference on Learning Representations https://openreview.net/forum?id=H1ezFREtwH (ICLR, 2020).

Sharma, A., Gu, S., Levine, S., Kumar, V. & Hausman, K. Dynamics-aware unsupervised discovery of skills. In Proc. International Conference on Learning Representations https://openreview.net/forum?id=HJgLZR4KvH (ICLR, 2020).

Tessler, C., Givony, S., Zahavy, T., Mankowitz, D. J. & Mannor, S. A deep hierarchical approach to lifelong learning in minecraft. In Proc. 31st AAAI Conference on Artificial Intelligence 1553–1561 (AAAI, 2017).

Vezhnevets, A. et al. Strategic attentive writer for learning macro-actions. In Neural Information Processing Systems ( NIPS ) (eds. Lee, D. et al.) 3494–3502 (NIPS, 2016).

Devin, C., Gupta, A., Darrell, T., Abbeel, P. & Levine, S. Learning modular neural network policies for multi-task and multi-robot transfer. In Proc. International Conference on Robotics and Automation ( ICRA ) (eds. Okamura, A. et al.) 2169–2176 (IEEE, 2017).

Hejna, D. J., Abbeel, P. & Pinto, L. Hierarchically decoupled morphological transfer. In Proc. International Conference on Machine Learning ( ICML ) (eds. Daumé III, H. & Singh, A.) 11409–11420 (PMLR, 2020).

Hamrick, J. B. et al. On the role of planning in model-based deep reinforcement learning. In Proc. International Conference on Learning Representations https://openreview.net/pdf?id=IrM64DGB21 (ICLR, 2021).

Sutton, R. S. Integrated architectures for learning, planning, and reacting based on approximating dynamic programming. In Proc. 7th International Conference on Machine Learning ( ICML ) (eds. Porter, B. W. & Mooney, R. J.) 216–224 (Morgan Kaufmann, 1990).

Nau, D. et al. SHOP2: an HTN planning system. J. Artif. Intell. Res. 20 , 379–404 (2003).

Article   MATH   Google Scholar  

Lyu, D., Yang, F., Liu, B. & Gustafson, S. SDRL: interpretable and data-efficient deep reinforcement learning leveraging symbolic planning. In Proc. AAAI Conference on Artificial Intelligence Vol. 33, 2970–2977 (AAAI, 2019).

Ma, A., Ouimet, M. & Cortés, J. Hierarchical reinforcement learning via dynamic subspace search for multi-agent planning. Auton. Robot. 44 , 485–503 (2020).

Bacon, P.-L., Harb, J. & Precup, D. The option-critic architecture. In Proc. 31st AAAI Conference on Artificial Intelligence 1726–1734 (AAAI, 2017).

Dietterich, T. G. State abstraction in MAXQ hierarchical reinforcement learning. In Advances in Neural Information Processing Systems ( NIPS ) (eds. Solla, S. et al.) Vol. 12, 994–1000 (NIPS, 1999).

Kulkarni, T. D., Narasimhan, K. R., Saeedi, A. & Tenenbaum, J. B. Hierarchical deep reinforcement learning: integrating temporal abstraction and intrinsic motivation. In Neural Information Processing Systems ( NIPS ) (eds. Lee, D. et al.) 3675–3683 (NIPS, 2016).

Shankar, T., Pinto, L., Tulsiani, S. & Gupta, A. Discovering motor programs by recomposing demonstrations. In Proc. International Conference on Learning Representations https://openreview.net/attachment?id=rkgHY0NYwr&name=original_pdf (ICLR, 2020).

Vezhnevets, A. S., Wu, Y. T., Eckstein, M., Leblond, R. & Leibo, J. Z. Options as responses: grounding behavioural hierarchies in multi-agent reinforcement learning. In Proc. International Conference on Machine Learning ( ICML ) (eds. Daumé III, H. & Singh, A.) 9733–9742 (PMLR, 2020).

Ghazanfari, B., Afghah, F. & Taylor, M. E. Sequential association rule mining for autonomously extracting hierarchical task structures in reinforcement learning. IEEE Access 8 , 11782–11799 (2020).

Levy, A., Konidaris, G., Platt, R. & Saenko, K. Learning multi-level hierarchies with hindsight. In Proc. International Conference on Learning Representations https://openreview.net/pdf?id=ryzECoAcY7 (ICLR, 2019).

Nachum, O., Gu, S., Lee, H. & Levine, S. Data-efficient hierarchical reinforcement learning. In Proc. 32nd International Conference on Neural Information Processing Systems (NIPS) (eds. Bengio, S. et al.) 3307–3317 (NIPS, 2018).

Rafati, J. & Noelle, D. C. Learning representations in model-free hierarchical reinforcement learning. In Proc. 33rd AAAI Conference on Artificial Intelligence 10009–10010 (AAAI, 2019).

Röder, F., Eppe, M., Nguyen, P. D. H. & Wermter, S. Curious hierarchical actor-critic reinforcement learning. In Proc. International Conference on Artificial Neural Networks ( ICANN ) (eds. Farkaš, I. et al.) 408–419 (Springer, 2020).

Zhang, T., Guo, S., Tan, T., Hu, X. & Chen, F. Generating adjacency-constrained subgoals in hierarchical reinforcement learning. In Neural Information Processing Systems ( NIPS ) (eds. Larochelle, H. et al.) 21579-21590 (NIPS, 2020).

Lample, G. & Chaplot, D. S. Playing FPS games with deep reinforcement learning. In Proc. 31st AAAI Conference on Artificial Intelligence 2140–2146 (AAAI, 2017).

Vezhnevets, A. S. et al. FeUdal networks for hierarchical reinforcement learning. In Proc. 34th International Conference on Machine Learning ( ICML ) (eds. Precup, D. & Teh, Y. W.) Vol. 70, 3540–3549 (PMLR, 2017).

Wulfmeier, M. et al. Compositional Transfer in Hierarchical Reinforcement Learning. In Robotics: Science and System XVI (RSS) (eds. Toussaint M. et al.) (Robotics: Science and Systems Foundation, 2020); https://arxiv.org/abs/1906.11228

Yang, Z., Merrick, K., Jin, L. & Abbass, H. A. Hierarchical deep reinforcement learning for continuous action control. IEEE Trans. Neural Netw. Learn. Syst. 29 , 5174–5184 (2018).

Toussaint, M., Allen, K. R., Smith, K. A. & Tenenbaum, J. B. Differentiable physics and stable modes for tool-use and manipulation planning. In Proc. Robotics : Science and Systems XIV ( RSS ) (eds. Kress-Gazit, H. et al.) https://ipvs.informatik.uni-stuttgart.de/mlr/papers/18-toussaint-RSS.pdf (Robotics: Science and Systems Foundation, 2018).

Akrour, R., Veiga, F., Peters, J. & Neumann, G. Regularizing reinforcement learning with state abstraction. In Proc. IEEE / RSJ International Conference on Intelligent Robots and Systems ( IROS ) 534–539 (IEEE, 2018).

Schaul, T. & Ring, M. Better generalization with forecasts. In Proc. 23rd International Joint Conference on Artificial Intelligence ( IJCAI ) (ed. Rossi, F.) 1656–1662 (AAAI, 2013).

Colas, C., Akakzia, A., Oudeyer, P.-Y., Chetouani, M. & Sigaud, O. Language-conditioned goal generation: a new approach to language grounding for RL. Preprint at https://arxiv.org/abs/2006.07043 (2020).

Blaes, S., Pogancic, M. V., Zhu, J. J. & Martius, G. Control what you can: intrinsically motivated task-planning agent. Neural Inf. Process. Syst. 32 , 12541–12552 (2019).

Haarnoja, T., Hartikainen, K., Abbeel, P. & Levine, S. Latent space policies for hierarchical reinforcement learning. In Proc. International Conference on Machine Learning ( ICML ) (eds. Dy, J. & Krause, A.) Vol. 4, 2965–2975 (PMLR, 2018).

Rasmussen, D., Voelker, A. & Eliasmith, C. A neural model of hierarchical reinforcement learning. PLoS ONE 12 , e0180234 (2017).

Riedmiller, M. et al. Learning by playing—solving sparse reward tasks from scratch. In Proc. International Conference on Machine Learning ( ICML ) (eds. Dy, J. & Krause, A.) Vol. 10, 6910–6919 (PMLR, 2018).

Yang, F., Lyu, D., Liu, B. & Gustafson, S. PEORL: integrating symbolic planning and hierarchical reinforcement learning for robust decision-making. In Proc. 27th International Joint Conference on Artificial Intelligence ( IJCAI ) (ed. Lang, J.) 4860–4866 (IJCAI, 2018).

Machado, M. C., Bellemare, M. G. & Bowling, M. A Laplacian framework for option discovery in reinforcement learning. In Proc. International Conference on Machine Learning (ICML) (eds. Precup, D. & Teh, Y. W.) Vol. 5, 3567–3582 (PMLR, 2017).

Pathak, D., Agrawal, P., Efros, A. A. & Darrell, T. Curiosity-driven exploration by self-supervised prediction. In Proc. 34th International Conference on Machine Learning ( ICML ) (eds. Precup, D. & Teh, Y. W.) 2778–2787 (PMLR, 2017).

Schillaci, G. et al. Intrinsic motivation and episodic memories for robot exploration of high-dimensional sensory spaces. Adaptive Behav. 29 549–566 (2020).

Colas, C., Fournier, P., Sigaud, O., Chetouani, M. & Oudeyer, P.-Y. CURIOUS: intrinsically motivated modular multi-goal reinforcement learning. In Proc. International Conference on Machine Learning ( ICML ) (eds. Chaudhuri, K. & Salakhutdinov, R.) 1331–1340 (PMLR, 2019).

Hafez, M. B., Weber, C., Kerzel, M. & Wermter, S. Improving robot dual-system motor learning with intrinsically motivated meta-control and latent-space experience imagination. Robot. Auton. Syst. 133 , 103630 (2020).

Yamamoto, K., Onishi, T. & Tsuruoka, Y. Hierarchical reinforcement learning with abductive planning. In Proc. ICML / IJCAI / AAMAS 2018 Workshop on Planning and Learning ( PAL-18 ) (2018).

Wu, B., Gupta, J. K. & Kochenderfer, M. J. Model primitive hierarchical lifelong reinforcement learning . In Proc. International Joint Conference on Autonomous Agents and Multiagent Systems ( AAMAS ) (eds. Agmon, N. et al.) Vol. 1, 34–42 (IFAAMAS, 2019).

Li, Z., Narayan, A. & Leong, T. Y. An efficient approach to model-based hierarchical reinforcement learning. In Proc. 31st AAAI Conference on Artificial Intelligence 3583–3589 (AAAI, 2017).

Hafner, D., Lillicrap, T. & Norouzi, M. Dream to control: learning behaviors by latent imagination. In Proc. International Conference on Learning Representations https://openreview.net/pdf?id=S1lOTC4tDS (ICLR, 2020).

Deisenroth, M. P., Rasmussen, C. E. & Fox, D. Learning to control a low-cost manipulator using data-efficient reinforcement learning. In Robotics : Science and Systems VII ( RSS ) (eds. Durrant-Whyte, H. et al.) 57–64 (Robotics: Science and Systems Foundation, 2011).

Ha, D. & Schmidhuber, J. Recurrent world models facilitate policy evolution. In Proc. 32nd International Conference on Neural Information Processing Systems (NeurIPS) (eds. Bengio, S. et al.) 2455–2467 (NIPS, 2018).

Battaglia, P. W. et al. Relational inductive biases, deep learning and graph networks. Preprint at https://arxiv.org/abs/1806.01261 (2018).

Andrychowicz, M. et al. Hindsight experience replay. In Proc. Neural Information Processing Systems ( NIPS ) (eds. Guyon I. et al.) 5048–5058 (NIPS, 2017); https://papers.nips.cc/paper/7090-hindsight-experience-replay.pdf

Schwartenbeck, P. et al. Computational mechanisms of curiosity and goal-directed exploration. eLife 8 , e41703 (2019).

Haarnoja, T., Zhou, A., Abbeel, P. & Levine, S. Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In Proc. International Conference on Machine Learning ( ICML ) (eds. Dy, J. & Krause, A.) 1861–1870 (PMLR, 2018).

Yu, A. J. & Dayan, P. Uncertainty, neuromodulation and attention. Neuron 46 , 681–692 (2005).

Baldwin, D. A. & Kosie, J. E. How does the mind render streaming experience as events? Top. Cogn. Sci. 13 , 79–105 (2021).

Download references

Acknowledgements

We acknowledge funding from the DFG (projects IDEAS, LeCAREbot, TRR169, SPP 2134, RTG 1808 and EXC 2064/1), the Humboldt Foundation and Max Planck Research School IMPRS-IS.

Author information

Manfred Eppe

Present address: Hamburg University of Technology, Hamburg, Germany

Authors and Affiliations

Universität Hamburg, Hamburg, Germany

Manfred Eppe, Matthias Kerzel, Phuong D. H. Nguyen & Stefan Wermter

University of Tübingen, Tübingen, Germany

Christian Gumbsch & Martin V. Butz

Max Planck Institute for Intelligent Systems, Tübingen, Germany

Christian Gumbsch

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Manfred Eppe .

Ethics declarations

Competing interests.

The authors declare no competing interests.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Supplementary information.

Supplementary Boxes 1–6 and Table 1.

Rights and permissions

Reprints and permissions

About this article

Cite this article.

Eppe, M., Gumbsch, C., Kerzel, M. et al. Intelligent problem-solving as integrated hierarchical reinforcement learning. Nat Mach Intell 4 , 11–20 (2022). https://doi.org/10.1038/s42256-021-00433-9

Download citation

Received : 18 December 2020

Accepted : 07 December 2021

Published : 25 January 2022

Issue Date : January 2022

DOI : https://doi.org/10.1038/s42256-021-00433-9

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

This article is cited by

Efficient stacking and grasping in unstructured environments.

• Jinbiao Zhu

Journal of Intelligent & Robotic Systems (2024)

Four attributes of intelligence, a thousand questions

• Matthieu Bardal
• Eric Chalmers

Biological Cybernetics (2023)

An Alternative to Cognitivism: Computational Phenomenology for Deep Learning

• Pierre Beckmann
• Guillaume Köstner
• Inês Hipólito

Minds and Machines (2023)

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

Sign up for the Nature Briefing: AI and Robotics newsletter — what matters in AI and robotics research, free to your inbox weekly.

• Graph Theory

Problem solving

There are N problems numbered 1..N which you need to complete. You've arranged the problems in increasing difficulty order, and the i th problem has estimated difficulty level i . You have also assigned a rating vi to each problem. Problems with similar vi values are similar in nature. On each day, you will choose a subset of the problems and solve them. You've decided that each subsequent problem solved on the day should be tougher than the previous problem you solved on that day. Also, to make it less boring, consecutive problems you solve should differ in their vi rating by at least K. What is the least number of days in which you can solve all problems?

Input Format

The first line contains the number of test cases T. T test cases follow. Each case contains an integer N and K on the first line, followed by integers v1,...,vn on the second line.

Constraints

1 <= T <= 100 1 <= N <= 300 1 <= vi <= 1000 1 <= K <= 1000

Output Format

Output T lines, one for each test case, containing the minimum number of days in which all problems can be solved.

Sample Input

Sample Output

Explanation

For the first example, you can solve the problems with rating 5 and 7 on the first day and the problem with rating 4 on the next day. Note that the problems with rating 5 and 4 cannot be completed consecutively because the ratings should differ by at least K (which is 2). Also, the problems cannot be completed in order 5,7,4 in one day because the problems solved on a day should be in increasing difficulty level.

For the second example, all problems can be solved on the same day.

Cookie support is required to access HackerRank

Seems like cookies are disabled on this browser, please enable them to open this website

• Your Security
• LinkLive Chat
• Measuring, Monitoring & Planning (MMP)
• Campaign Manager
• Knowledge Base
• Mobile Worker
• Messaging Bundle
• Engagement Bundle
• Advanced Bundle
• REVA AI Healthcare
• Reva AI Banking
• All Resources
• Case Studies
• Healthcare Scenarios
• Financial Scenarios
• Customer Success

Level Zero Solvable

Share This Story, Choose Your Platform!

About the author: revation systems.

• DNIS (Dialed Number Identification Service)
• Full Coverage Scheduling
• Application Programming Interface (API)
• Index Factor

Uniform Call Distribution (UCD)

Stakeholder.

• Calls Per Agent

Subscribe for Updates

Contact sales.

• Phone: 1.952.392.1831
• Email: [email protected]

Recent Posts

Posts by topic.

• Glossary (15)

Revation Systems

Providing secure, compliant unified communications via our cloud platform. From encrypted email to secure desktop sharing to secure voice and video collaboration, Revation LinkLive is the first truly unified communications platform.

• Revation Home
• Request a Demo
• Compare Bundles
• Financial Services
• Digital Transformation
• Customer Experience
• Unified Communications
• In the News
• Subscribe to Updates
• Sales ( Email )
• Sales ( Phone )