A Beginner’s Guide to PicoCTF’s Reverse Engineering: Simple Writeups

Welcome to the world of reverse engineering!
This crucial skill helps cybersecurity experts safeguard systems and discover vulnerabilities.
In this post, we dive into picoCTF, a captivating platform for learning reverse engineering.
Building on our previous articles, “PicoCTF Unlocked: Mastering Cybersecurity One Step at a Time” and “Unravelling the Secrets of Reverse Engineering: Practical Applications for In-Depth Analysis“, we now guide you through solving three simple picoCTF challenges in the reverse engineering section.
Perfect for beginners, these practical examples will boost your cybersecurity journey. Let’s jump in and start exploring!

Getting started with picoCTF

PicoCTF is an engaging, interactive platform designed for learning cybersecurity through Capture The Flag (CTF) challenges. Aimed at beginners, picoCTF offers a gamified approach to enhance cybersecurity skills in various domains, including reverse engineering.

First, let’s get started with it. (For a more detailed description of how to register and solve a challenge in the “Practice” section you can read our previous article at this link).
Follow these simple steps to set up your account and environment:

1. Visit the picoCTF website and create an account. Fill in the required information and confirm your email.
2. Log in to your account and explore the dashboard. Familiarize yourself with the interface and available features.
3. Use the “Practice” section
4. Join or create a team. Invite friends or join other learners to collaborate and solve challenges together.
5. Choose your first challenge. Start with easier tasks and gradually progress to more complex ones.
6. Keep track of your progress. The dashboard displays your achievements and allows you to monitor your improvement.

Remember to have fun while learning and don’t be afraid to ask for help. The picoCTF community is always ready to support you.

Introduction to reverse engineering

Reverse engineering is the process of deconstructing a system, software, or hardware to understand its inner workings. In cybersecurity, this skill helps experts uncover vulnerabilities, develop effective defences, and analyze malicious code. It plays a crucial role in maintaining the security and integrity of digital systems.
For those new to those concepts, we recommend starting with our introductory article on Reverse Engineering found at https://www.stackzero.net/reverse-engineering/ that will target specifically reverse engineering on executables.
With a solid foundation, you’re ready to dive into our practical examples and begin mastering reverse engineering!

Now, let’s delve into reverse engineering within the realm of picoCTF. Reverse engineering involves breaking down a system to understand its inner workings. This process aids cybersecurity experts in identifying vulnerabilities and enhancing protection measures.

PicoCTF offers reverse engineering challenges that assess your abilities in analyzing and deciphering files or programs. These challenges come in diverse types and difficulty levels. As you tackle more tasks, your reverse engineering skills will improve, preparing you for real-life cybersecurity scenarios.

Tips and resources for mastering reverse engineering

Mastering reverse engineering requires dedication, practice, and access to the right resources. Here are some tips and resources to help you on your journey:

1. Build a strong foundation in programming languages, such as C, C++, Python, and Assembly. Understanding the underlying logic and structures will make reverse engineering tasks more manageable.
2. Familiarize yourself with different file formats and their structures. Knowing the ins and outs of various formats will aid in identifying hidden data and vulnerabilities.
3. Explore reverse engineering tools like Ghidra, IDA Pro, OllyDbg, and Radare2. Learn their capabilities, strengths, and weaknesses to choose the right tool for each task.
4. Engage with online communities and forums dedicated to reverse engineerings, such as GitHub, Stack Overflow, and Reddit. Exchanging knowledge and experiences with fellow learners will foster growth and problem-solving skills.
5. Enrol in online courses, read books and follow tutorials to deepen your understanding of reverse engineering concepts and techniques.

By following these tips and consistently applying your newfound knowledge, you’ll be on your way to mastering reverse engineering and becoming a skilled cybersecurity professional.

Asm1 challenge in picoCTF

We want to start the static analysis of the challenge called asm1, so in the section “reverse engineering” of picoCTF search for that.

So after finding it, download the file and let’s try to understand how it works!

The objective is to understand the behaviour of the given assembly code.

Here is the assembly code for asm-1 challenge:

asm1:
	<+0>:	push   ebp
	<+1>:	mov    ebp,esp
	<+3>:	cmp    DWORD PTR [ebp+0x8],0x3a2
	<+10>:	jg     0x512 <asm1+37>
	<+12>:	cmp    DWORD PTR [ebp+0x8],0x358
	<+19>:	jne    0x50a <asm1+29>
	<+21>:	mov    eax,DWORD PTR [ebp+0x8]
	<+24>:	add    eax,0x12
	<+27>:	jmp    0x529 <asm1+60>
	<+29>:	mov    eax,DWORD PTR [ebp+0x8]
	<+32>:	sub    eax,0x12
	<+35>:	jmp    0x529 <asm1+60>
	<+37>:	cmp    DWORD PTR [ebp+0x8],0x6fa
	<+44>:	jne    0x523 <asm1+54>
	<+46>:	mov    eax,DWORD PTR [ebp+0x8]
	<+49>:	sub    eax,0x12
	<+52>:	jmp    0x529 <asm1+60>
	<+54>:	mov    eax,DWORD PTR [ebp+0x8]
	<+57>:	add    eax,0x12
	<+60>:	pop    ebp
	<+61>:	ret    

To reverse engineer this code, it is essential to understand the calling convention and the syntax used in the assembly language(You can find a good resource here). In this case, the code employs the x86 assembly language and uses a standard CDECL calling convention. The CDECL calling convention dictates that the caller cleans the stack and the function’s return value is stored in the EAX register. Now, let’s analyze the code step-by-step:

• The function starts by setting up the stack frame using ‘push ebp’ and ‘mov ebp, esp’.
• The first comparison checks if the input value is greater than 0x3a2. If so, it jumps to <+37>.
• The next comparison checks if the input value is equal to 0x358.
• If so, it proceeds to <+21>, where 0x12 is added to the input value, and jumps to <+60>.
• If the input value is not equal to 0x358, the code continues to <+29>, where 0x12 is subtracted from the input value and jumps to <+60>.
• At <+37>, another comparison is made to check if the input value is equal to 0x6fa.
• If so, it follows the same steps as in <+29>.
• If the input value is not equal to 0x6fa, it proceeds to <+54>, where 0x12 is added to the input value.
• Finally, the function concludes by restoring the stack frame and returning the value in EAX register.

The Real example

By examining the assembly code and grasping its logic, we can determine the function’s behaviour and output based on the input value. In our example, the input is 0x6fa, so according to the CDECL convention, it is placed into the stack at the address ebp+8 (the return address is at ebp+4).

We follow the path, and at the first comparison, we jump to <+37> because 0x6fa is greater than 0x358.
At this address, another comparison takes place, comparing 0x6fa with 0x6fa.
Since the values are equal, the JNE (jump not equal) instruction is not executed, and we subtract 0x12 from our input.
The final result is 0x6e8, which is our flag!

Enter our input, and we pass the level!

Asm2 challenge in picoCTF

Now, let’s dive into the second challenge in the reverse engineering section of picoCTF practice called asm2. Like the previous challenge, our goal is to understand the behaviour of the provided assembly code.

Here’s the code for asm2:

asm2:
	<+0>:	push   ebp
	<+1>:	mov    ebp,esp
	<+3>:	sub    esp,0x10
	<+6>:	mov    eax,DWORD PTR [ebp+0xc]
	<+9>:	mov    DWORD PTR [ebp-0x4],eax
	<+12>:	mov    eax,DWORD PTR [ebp+0x8]
	<+15>:	mov    DWORD PTR [ebp-0x8],eax
	<+18>:	jmp    0x509 <asm2+28>
	<+20>:	add    DWORD PTR [ebp-0x4],0x1
	<+24>:	add    DWORD PTR [ebp-0x8],0x74
	<+28>:	cmp    DWORD PTR [ebp-0x8],0xfb46
	<+35>:	jle    0x501 <asm2+20>
	<+37>:	mov    eax,DWORD PTR [ebp-0x4]
	<+40>:	leave  
	<+41>:	ret    

In this challenge, we follow these steps to reverse engineer the assembly code:

1. The function starts by setting up the stack frame and allocating 0x10 bytes of memory using ‘push ebp’, ‘mov ebp, esp’, and ‘sub esp, 0x10’.
2. It then moves the second function argument (DWORD PTR [ebp+0xc]) into the EAX register and stores it in the local variable (DWORD PTR [ebp-0x4]).
3. Next, it moves the first function argument (DWORD PTR [ebp+0x8]) into the EAX register and stores it in another local variable (DWORD PTR [ebp-0x8]).
4. The code proceeds to <+18> and jumps to <+28>.
5. At <+28>, it compares the value stored at [ebp-0x8] with 0xfb46. If the value is less than or equal to 0xfb46, it jumps back to <+20>.
6. At <+20>, it increments the value at [ebp-0x4] by 0x1 and adds 0x74 to the value at [ebp-0x8]. Then, it loops back to the comparison at <+28>.
7. Once the value at [ebp-0x8] is greater than 0xfb46, it moves to <+37>, where the value stored at [ebp-0x4] is moved into the EAX register.
8. Finally, the function releases the stack frame and returns the value in the EAX register.

The Real example

In this example, following the CDECL calling convention, the value 0x4 is at address ebp+0x8, and the value 0x21 is at address ebp+0xc. Using the EAX register, these values are moved into local variables:

• 0x21 -> ebp-0x4
• 0x4 -> ebp-0x8

The value in ebp-0x8 is then compared to 0xfb46, and it increases until it is less than or equal.
For each iteration, the loop adds 0x1 to the value in ebp-0x4 and 0x74 to the value in ebp-0x8. To clarify and simplify the understanding, I’ll write a short Python script to replicate the operations and obtain the result:

var_4 = 0x21
var_8 = 0x4

while var_8 < 0xfb46:
    var_4 += 0x1
    var_8 += 0x74

print(hex(var_4))

Running the code, we get the value 0x24c, which is our key, and we pass the level!

Conclusion

Obviously, we could compile, run and see the results, but it wouldn’t help us to understand, so I hope you appreciated the effort.

In conclusion, reverse engineering is an essential skill in cybersecurity, and picoCTF provides a fun and engaging way to learn. By practising with the practical examples we’ve discussed, you’re taking your first steps towards becoming a reverse engineering expert.

We encourage you to continue exploring and honing your skills. Remember, perseverance and curiosity are your greatest assets in this journey. For more insights and resources, follow our blog at stackzero.net and connect with us on our social media profiles. Together, let’s build a safer digital world!