• Uncategorized

About python : Folder-seems-to-exist-in-OS-but-fails-silently-upon-sudo-rm–rf-dirname

Question Detail

I am trying to delete a folder, but failing to do so. I am sudo, and have all permissions.

cd /path/to/parent

ls

001  003  005  007  009  011  013  015  017  019  021                 dent_detections_3d.ply      dent_spread_debug.png             paint_chip_spread_debug.png
002  004  006  008  010  012  014  016  018  020  _det__grouped.json  dent_detections_debug.json  paint_chip_detections_debug.json

I want to delete 003 which is the problematic folder. Other folders behave normally.

Observations:

  1. cd 003 gives -bash: cd: 003: No such file or directory
  2. sudo rm -rf 003 or sudo rm -rf 003/ gives empty output, then ls gives the same as above (003 exists).
python
> import os
> path.os.exists("path/to/parent")
>> True
> path.os.exists("path/to/parent/003")
>> False
> os.mkdir("path/to/parent/003")
>> 
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [Errno 17] File exists: '/path/to/parent/003'

What may be causing this? How to debug this? I want the folder removed, and I need this to not happen again.


EDIT to answer comments:

> ls -la

total 6144
drwxr-xr-x 2 noam root      0 Dec 28 13:28 .
drwxr-xr-x 2 noam root      0 Dec  9 18:41 ..
drwxr-xr-x 2 noam root      0 Dec 28 13:28 001
drwxr-xr-x 2 noam root      0 Dec 28 13:28 002
drwxr-xr-x 2 noam root      0 Dec 28 13:28 003
drwxr-xr-x 2 noam root      0 Dec 28 13:26 004
drwxr-xr-x 2 noam root      0 Dec 28 13:26 005
drwxr-xr-x 2 noam root      0 Dec 28 13:26 006
drwxr-xr-x 2 noam root      0 Dec 28 13:26 007
drwxr-xr-x 2 noam root      0 Dec 28 13:26 008
drwxr-xr-x 2 noam root      0 Dec 28 13:26 009
drwxr-xr-x 2 noam root      0 Dec 28 13:26 010
drwxr-xr-x 2 noam root      0 Dec 28 13:26 011
drwxr-xr-x 2 noam root      0 Dec 28 13:26 012
drwxr-xr-x 2 noam root      0 Dec 28 13:26 013
drwxr-xr-x 2 noam root      0 Dec 28 13:26 014
drwxr-xr-x 2 noam root      0 Dec 28 13:26 015
drwxr-xr-x 2 noam root      0 Dec 28 13:26 016
drwxr-xr-x 2 noam root      0 Dec 28 13:26 017
drwxr-xr-x 2 noam root      0 Dec 28 13:26 018
drwxr-xr-x 2 noam root      0 Dec 28 13:26 019
drwxr-xr-x 2 noam root      0 Dec 28 13:26 020
drwxr-xr-x 2 noam root      0 Dec 28 13:26 021
-rwxr-xr-x 1 noam root 684753 Dec 12 11:58 _det__grouped.json
-rwxr-xr-x 1 noam root   7604 Dec 12 11:58 dent_detections_3d.ply
-rwxr-xr-x 1 noam root  89902 Dec 12 11:58 dent_detections_debug.json
-rwxr-xr-x 1 noam root 377863 Dec 12 11:58 dent_spread_debug.png
-rwxr-xr-x 1 noam root     24 Dec 12 11:58 paint_chip_detections_debug.json
-rwxr-xr-x 1 noam root 362525 Dec 12 11:58 paint_chip_spread_debug.png


EDIT2 with the information that a race-condition is the problem

Relevant code chunk that creates the folder:

if os.path.exists(cluster_folder_path):
    shutil.rmtree(cluster_folder_path)
os.mkdir(cluster_folder_path)

Question Answer

Changing the code in EDIT2 to

while os.path.exists(cluster_folder_path):
    shutil.rmtree(cluster_folder_path)
os.mkdir(cluster_folder_path)

forced the folder to actually be deleted before moving on to create it, thus preventing the race condition.

I was not able to understand why this happens only for a single folder, or why this one and not others. I was also not able to understand why this kept happening after Python was already not running.

Will accept an answer that manages to explain that.

You may also like...

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.