How to Organize Files and Folders Using Python | by Sahil Fruitwala | Jun, 2022

3 main things will help us decide how we want to organize our files

Photo by JF Martin on Unsplash

3 main things will help us decide how we want to organize our files.

  1. How do we want to manage files?
  2. What kind of files do we want to organize?
  3. Which directory do we want to organize?

How do we want to organize files?

We can organize files based on various factors such as name, creation date, modified date and size. We can also manage files based on the type/extension of a file. That is where the next question comes into the picture.

What kind of files do we want to organize?

There are many types of files we might find in our download folder. There can be music, video, images, zip, executable, document, and many more. After answering the first two questions, we must decide which directory we want to organize.

This question is important if you are organizing the subdirectories as well. For example, there can be a coding project that can contain many kinds of files. To organize this directory based on file type is not a good idea because it can mess up our whole project.

So for this tutorial, we will mostly focus on the root directory.

To organize any directory, first, we need a list of files. To get the list of all files in the given directory we can use the listdir() method from the os module.

Syntax: os.listdir(path)

import ostotal_list = os.listdir("Downloads")
print(total_list)
""" OUTPUT:['elements.svg', 'Export.zip', 'favicon_io', 'Frame 5.png',
'Screenshot.png', 'Temp', 'Video-1.webm']
"""

Here, we get a list that contains files and directories both.

Screenshot of download folder
Screenshot of the download folder

If we want to use this, then we have to separate files and directories by ourselves using the os.path. To identify if the given path is a file or directory we can use the following code snippet:

from os import listdir, pathtotal_list = listdir(".")file_list = []
dir_list = []
for file in total_list:
if path.isfile(file):
file_list.append(file)
else:
dir_list.append(file)
print(file_list)
print("-"*15)
print(dir_list)
""" OUTPUT:
['elements.svg', 'Export.zip', 'Frame 5.png', 'Screenshot.png', 'Video-1.webm']
---------------
['favicon_io', 'Temp']
"""

When I run this code, I was in the Download directory. So, if I want a list from the current directory, I can use os.listdir("./") or os.listdir(".") or os.listdir().

You might say this is a lot of work just to get a list of filenames. So, let me present to you an easy way. I did not know about this method as well. I found it when I was researching for this tutorial. We can use the os.walk() method to retrieve all the files. You can check out the blog post here from where I got to learn about this.

Syntax: os.walk(path)

from os import walkfiles = []
for (dirpath, dirnames, filenames) in walk("./"):
files.extend(filenames)
break
print(filenames)""" OUTPUT:
['elements.svg', 'Export.zip', 'Frame 5.png',
'Screenshot.png', 'temp.py', 'Video-1.webm']
"""

First of all, let me explain why I use the break keyword here. The walk() method not only lists out directories and files of the given location. But, it tries to scan all the subdirectories as well. Let me show you.

from os import walkfiles = []
for (index, data) in enumerate(walk("./")):
if index == 2:
break
print(index)
print(data)
print("-"*15)
""" OUTPUT:0
('./', ['favicon_io', 'Temp'], ['elements.svg', 'Export.zip', 'Frame 5.png', 'Screenshot.png', 'temp.py', 'Video-1.webm'])
---------------
1
('./favicon_io', ['2021', '2022'], ['android-chrome-192x192.png', 'android-chrome-512x512.png', 'apple-touch-icon.png', 'favicon-16x16.png', 'favicon-32x32.png', 'favicon.ico', 'site.webmanifest'])
---------------
"""

As you can see, the first iteration yields the current path and two lists**.** The first list contains all the directories and the second list contains all the file names. Now, during the second iteration, the walk() method starts scanning the subdirectories which are “favicon_io” and “Temp” in our case. We are currently focusing only on the “Downloads” directory. So, I use the break keyword so that after one iteration it stops scanning subdirectories.

Screenshot of favicon subdirectory
Screenshot of favicon subdirectory

Once we got the list of all files, we need to separate them based on file type. To do that we will use the extension of each file.

To get the file extension, we can use the path module from the os. The path module has a splitext() method that will return a tuple containing the file name and its extension.

Syntax: os.path.splitext(path)

from os import pathfile_tuple = path.splitext('Screenshot.jpg')
print(file_tuple)
print(file_tuple[1])
""" OUTPUT:
('Screenshot', '.jpg')
.jpg
"""

Now, we have all files and we know how to get the extension. So, the next part is to move files according to their types.

We can move files with the os module but this time we will use shutil which I think is way better than os for this task. We will use the move() method from the shutil module.

Syntax: shutil.move(src, dst, copy_function=copy2)

Note: The third argument is optional.

import shutilshutil.move("Screenshot.png", "Temp/")
shutil.move("elements.svg", "Temp")
shutil.move("Frame 5.png", "Temp/Frame 5.png")

We can move files three ways using shutil.move()

Last and final step. We need to create a new folder if it does not exist to move all files with the same type.

We can create a folder using the os.mkdir() method.

Syntax: os.mkdir(path, mode=0o777, *, dir_fd=None)

Note: The mode and dir_fd arguments are optional.

from os import mkdir, pathif not path.exists("Temp"):
mkdir("Temp")

The path.exists() method checks if the given location exists or not. We need to validate the existence of the directory because if a directory already exists, mkdir() will throw an error. And we want to create a directory only if it does not exist.

Now, as we have gone throw all the steps. It’s time to put them together.

To recap, we need to take the following steps:

  1. Get a list of files
  2. Extract extension for each file
  3. Create a folder if it does not exist
  4. Move files to the respective folder
from os import mkdir, path, walk
from pathlib import Path as pathlib
from shutil import move
IMAGE_EXT = [".jpg", ".png", ".jpeg", ".gif", ".webp", ".eps"]
DOWNLOAD_LOCATION = str(pathlib.home() / "Downloads")
file_list = []
for (dirpath, dirnames, filenames) in walk(DOWNLOAD_LOCATION):
file_list.extend(filenames)
break
def check_directory(dir_location):
"""Check if directory exists or not. If not create one."""
if not path.exists(dir_location):
mkdir(dir_location)
def file_mover(extention_list, new_location):
"""move file based on extension list to new location."""
for file in file_list:
if (path.splitext(file)[1]).lower() in extention_list:
move(file, new_location)
def organize_images():
"""Organize images based IMAGE_EXT list"""
image_location = path.join(DOWNLOAD_LOCATION, "Organized Images")
check_directory(image_location)
file_mover(IMAGE_EXT, image_location)
organize_images()

Here, most of the code is from earlier or very common. The one difference that you might see is that I have used the pathlib module. To make a generic code that can work with Linux, Windows and macOS, I used the pathlib module. It can give me the direct path to the default home directory for every operating system. I have tested it on Ubuntu, Windows and macOS.

I have tried to keep it simple as much as I could. In this example, I have shown how to organize image files. But you can do it for all kinds of files. It will be hard to read and understand here that is why I have pushed code that you can use for any type of file on my GitHub.

It is not difficult to operate and use files and directories in Python. If you know what you want to do and what you are looking for you can find almost everything that you need by googling. That is how I came across the walk() method.

If you want to learn more about Python, check out my ongoing Python series.

That was it. Thank you for reading. Till the next time 👋

References

  1. https://pynative.com/python-list-files-in-a-directory/
  2. https://www.geeksforgeeks.org/how-to-move-files-and-directories-in-python
Want to Connect?I know it is a lot, but, I hope, you were able to absorb some knowledge. Let me know if you need any help or want to discuss something. Reach out to me on Twitter or LinkedIn. Make sure to share any thoughts, questions, or concerns. I would love to see them.

Leave a Comment