[Yaffs] Unexpected erase behavior when rewriting a file on a nearly empty YAFFS

[Charles Manning]

Peter

Thanx for the feedback. You're essentially describing a proposed change to the 
garbage collection policy.

I quite like the idea. It has slight/low impact on NAND, but could be rather 
material to NOR folks.

More below:

On Saturday 05 November 2005 06:23, Peter Barada wrote:
> I've got YAFFS working on a NOR-based system, and I'm doing some testing
> on it.  If I start with an erased YAFFS, mount it on /mnt/yaffs, and
> then:
>
> # cp /bin/bash /mnt/yaffs
> # cp /bin/bash /mnt/yaffs
>
> I see that calls are made to erase the block(s).  I'm wondering why?  I
> thought that when chunks are overwritten, that they are marked as
> available for recycling and leave it to the garbage collector to do it.

To recap on how garbage collection works. There are two parts to it:

A) Erasure is kicked off as the result of the last chunk in a full block 
becoming unused. We only look at full blocks because we don't want to nuke 
the current block being written.

B) The rest of the garbage collection mechanism is there to sweep  data so 
that space can be reclaimed when the in-use data is sparsely distributed over 
the NAND. The way this works is by picking a block with a lot of dirt in it 
then copying off the useful stuff and deleting the original. As soon as the 
last chunk of useful stuff is copied off, (A) above hapens and the block is 
recycled.


To further complicate the story, there's "regular deletion" and "soft 
deletion":
Reglar deletion was at one time the only deletion mechanism. When a chunk is 
no longer needed, it is marked for deletion and if this caused the pagesinuse 
count to get to zero then (A) happens.

When you delete a file, the above would happen a lot, so "soft deletion" was 
introduced. This uses a quick marking method to mark all the chunks in the 
file without actually programming their deletion markers. The whole file is 
marked as deleted and thus this can only be done at a file granularity.

[This is applicable to yaffs1 handling. ie YAFFS1 or YAFFS2 with yaffs1 
compatability mode. YAFFS2 never writes deletion markers].

When you do 
 # cp /bin/bash /mnt/yaffs
 # cp /bin/bash /mnt/yaffs
 the second cp results in a open with truncation. This does not use soft 
deletion, but uses full deletion instead because the whole file is not being 
deleted.

If instead you did:
 # cp /bin/bash /mnt/yaffs
# rm /mnt/yaffs/bash
 # cp /bin/bash /mnt/yaffs

The rm would cause the whole file to be deleted and thus soft delete would be 
used instead. That would save the programming of deletion markers and would 
spread the erasures through garbage collection cycles.

The above could also be done by the special "resize to zero" handling ideas 
that have been semi-proposed and remain on the list of cool things worth 
doing.

>
> The callback from yaffs_EraseBlockInNand looks like:
>
> (gdb) where 5
> #0  yaffs_EraseBlockInNAND (dev=0xc06a7000, blockInNAND=13)
>     at yaffs_guts.c:352
> #1  0xc010ee16 in yaffs_BlockBecameDirty (dev=0xc06a7000, blockNo=13)
>     at yaffs_guts.c:2118
> #2  0xc010fa56 in yaffs_DeleteChunk (dev=0xc06a7000, chunkId=0,
> markNAND=1)
>     at yaffs_guts.c:2935
> #3  0xc01103e4 in yaffs_ResizeFile (in=0xc037c132, newSize=0)
>     at yaffs_guts.c:3695
> #4  0xc0111374 in yaffs_SetAttributes (obj=0xc037c132, attr=0xc06a7000)
>     at yaffs_guts.c:4627
> (More stack frames follow...)
>
> I think the issue is the test:
>
> 		if(bi->pagesInUse == 0 &&
> 	       bi->blockState == YAFFS_BLOCK_STATE_FULL)
> 	    {
> 	    	yaffs_BlockBecameDirty(dev,block);
> 	    }
>
> in yaffs_DeleteChunk.  I *think*(but amd not sure) that if this test is
> disabled, the block with all its chunks marked dirty will be left alone
> until *all* the chunks are written and a garbage collection is forced.

Essentially, this (plus probably a few more tweaks) would defer the erasures 
until the gc.

There are some interesting issues that would have to be dealt to. For instance 
making sure that we never run the free blocks dry.

>
> I also think this will spread out writes even more in the filesystem.

It would spread out erases, I think is what you mean.
>
> Thoughts?

Probably a good idea. Definitely for NOR folks.

Really, the best for NOR is to use asynchronous erasure and erase suspend. ie.
instead of BlockBecameDirty() doing the erase inline, it instead put the block 
into a queue for erasure. This would stop th fs stalling on erases as it 
currently does.


-- Charles