Question about ARM Linux MM

[Ho Lee]

Hi,

I have a question about ARM Linux memory management. While I read over
the "Memory Managment Unit" in <ARM Architecture Reference Manual>,
I found that there is no PRESENT bit, ACCESSED bit and DIRTY bit in
the page descriptor that x86 architecture has. I checked the
implementation of cpu_xxx_set_pte() function, and it looks like ignoring
LPTE_PRESENT, LPTE_YOUNG, and LPTE_DIRTY flags.

I guess without ACCESSED flag, it's not possible to make use of LRU
algorithm
to make free pages, and without DIRTY flag, Linux can not distinguish the
changed pages from other unchanged writable pages to decide what page to
swap out. Of course kernel works without these bits, but I wonder it
could make it inefficient.

I would like to know whether I understood correctly, and what ARM Linux
do to get over such limitations.

Regards,
Ho

-------------------------------------------------------------------
Subscription options: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ/Etiquette: http://www.arm.linux.org.uk/armlinux/mailinglists.php

[Russell King - ARM Linux]

Post by Ho Lee
I have a question about ARM Linux memory management. While I read over
the "Memory Managment Unit" in <ARM Architecture Reference Manual>,
I found that there is no PRESENT bit, ACCESSED bit and DIRTY bit in
the page descriptor that x86 architecture has. I checked the
implementation of cpu_xxx_set_pte() function, and it looks like ignoring
LPTE_PRESENT, LPTE_YOUNG, and LPTE_DIRTY flags.
...
I would like to know whether I understood correctly, and what ARM Linux
do to get over such limitations.

We do use present, young and dirty bits (this is actually the reason why
we end up with a "Linux PTE" and a "hardware PTE". We encode the hardware
PTE from the Linux version in the cpu_xxx_set_pte() function. These
functions are carefully coded to be efficient, so they're not the most
readable of assembly.

However, they do take account of the three flags you mention above. If
they don't, then its buggy.

-------------------------------------------------------------------
Subscription options: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ/Etiquette: http://www.arm.linux.org.uk/armlinux/mailinglists.php

[Ho Lee]

Hi Russell,

Thank you for your comment, but I'm still kind of coufused. I do
understand "Linux PTE" and "hardware PTE" are different, and in
Linux side there are those flags. But if ARM MMU didn't support
that bits, ACCESSED and DIRTY flags wouldn't be updated by hardware,
and that flags in pte_t structure would be meaningless. I wonder
how to keep the "Linux PTE" up-to-date without hardware support.

Regards,
Ho

-----Original Message-----
From: Russell King - ARM Linux [mailto:***@arm.linux.org.uk]
Sent: Friday, June 06, 2003 11:08 AM
To: Ho Lee
Cc: linux-arm-***@lists.arm.linux.org.uk
Subject: Re: Question about ARM Linux MM

We do use present, young and dirty bits (this is actually the reason why
we end up with a "Linux PTE" and a "hardware PTE". We encode the hardware
PTE from the Linux version in the cpu_xxx_set_pte() function. These
functions are carefully coded to be efficient, so they're not the most
readable of assembly.

However, they do take account of the three flags you mention above. If
they don't, then its buggy.

-------------------------------------------------------------------
Subscription options: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ/Etiquette: http://www.arm.linux.org.uk/armlinux/mailinglists.php

[Russell King - ARM Linux]

Post by Ho Lee
Thank you for your comment, but I'm still kind of coufused. I do
understand "Linux PTE" and "hardware PTE" are different, and in
Linux side there are those flags. But if ARM MMU didn't support
that bits, ACCESSED and DIRTY flags wouldn't be updated by hardware,
and that flags in pte_t structure would be meaningless. I wonder
how to keep the "Linux PTE" up-to-date without hardware support.

You emulate them by taking faults. This is pretty common when dealing
with RISC CPUs. Also note that ARM effectively has "present" - the LSB
two bits = 00 will cause a fault.

-------------------------------------------------------------------
Subscription options: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ/Etiquette: http://www.arm.linux.org.uk/armlinux/mailinglists.php

[Ho Lee]

Got it! So at first we set the PTEs without any permissions,
and mark young flag at read fault and dirty flag at write fault.

I looked inside the cpu_xxx_set_pte() function again, and found that
It forces "Hardware PTE" invalid if young flag is not set. But
I'm curious about following code common for many architecture.
(in normal 2.4.20 source tree)

tst r1, #LPTE_WRITE | LPTE_DIRTY @ Write and Dirty?
orreq r2, r2, #HPTE_AP_WRITE

It marks the PTE writable if either the page is writable or dirty.
I think it should use 'teq' instruction instead of 'tst' instruction
to mark the page writable only after dirty flag is set.

Regards,
Ho

-----Original Message-----
From: Russell King - ARM Linux [mailto:***@arm.linux.org.uk]
Sent: Friday, June 06, 2003 11:30 AM
To: Ho Lee
Cc: linux-arm-***@lists.arm.linux.org.uk
Subject: Re: Question about ARM Linux MM

You emulate them by taking faults. This is pretty common when dealing
with RISC CPUs. Also note that ARM effectively has "present" - the LSB
two bits = 00 will cause a fault.

-------------------------------------------------------------------
Subscription options: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ/Etiquette: http://www.arm.linux.org.uk/armlinux/mailinglists.php

[Russell King - ARM Linux]

Post by Ho Lee
Got it! So at first we set the PTEs without any permissions,
and mark young flag at read fault and dirty flag at write fault.
I looked inside the cpu_xxx_set_pte() function again, and found that
It forces "Hardware PTE" invalid if young flag is not set. But
I'm curious about following code common for many architecture.
(in normal 2.4.20 source tree)
orreq r2, r2, #HPTE_AP_WRITE
It marks the PTE writable if either the page is writable or dirty.
I think it should use 'teq' instruction instead of 'tst' instruction
to mark the page writable only after dirty flag is set.

Careful. You missed this:

eor r1, r1, #L_PTE_PRESENT | L_PTE_YOUNG | L_PTE_WRITE | L_PTE_DIRTY

which inverts the test; we only set HPTE_AP_WRITE if _both_ L_PTE_WRITE
and L_PTE_DIRTY were set. Remember - tst only sets the Z flag when
(op1 & op2) is zero.

What we're doing is ((~op1) & op2) == 0 which is directly equivalent to
(op1 & op2) == op2.

-------------------------------------------------------------------
Subscription options: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ/Etiquette: http://www.arm.linux.org.uk/armlinux/mailinglists.php

[Ian Molton]

On Fri, 6 Jun 2003 13:07:32 -0700

Post by Ho Lee
Got it! So at first we set the PTEs without any permissions,
and mark young flag at read fault and dirty flag at write fault.

Oooh, thanks. I have been learning the ARM MMU stuff lately too and you
just gave me one of those 'enlightened' moments.

one question I have...

if you have a page which is read, its marked young. I assume its marked
read-only when the fault happens?

this way, when a write happens, the page will fault again, thus
becomming both young and dirty? (or does it just become dirty since
write implies read on ARM?)

thus only dirty pages must ever be flushed?
--
Spyros lair: http://www.mnementh.co.uk/
Do not meddle in the affairs of Dragons, for you are tasty and good with
ketchup.

Systems programmers keep it up longer.

-------------------------------------------------------------------
Subscription options: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ/Etiquette: http://www.arm.linux.org.uk/armlinux/mailinglists.php

[Russell King - ARM Linux]

Post by Ian Molton
one question I have...

I count three. 8)

Post by Ian Molton
if you have a page which is read, its marked young. I assume its marked
read-only when the fault happens?

If the page is marked read and young from the Linux perspective, it will
be marked read only to the MMU. CPU reads don't fault, CPU writes will.

Post by Ian Molton
this way, when a write happens, the page will fault again, thus
becomming both young and dirty? (or does it just become dirty since
write implies read on ARM?)

You'll get a fault. What you do in that situation is farly complex, but
for the sake of argument, lets say that we also have write permission from
the Linux perspective, but the dirty bit is clear.

The generic fault handling code in linux/mm/memory.c (handle_pte_fault)
notices that the page is present, that you wanted write access to the
page, and the page has write permissions. It's solution is to set the
dirty bit, and our response to that is to allow write accesses from
user space.

(ps, the important thing here is the permissions from user mode for
user space pages, not SVC mode. SVC mode accesses are treated
specially via the put_user/get_user/copy_{to,from}_user functions/macros.)

Post by Ian Molton
thus only dirty pages must ever be flushed?

This is a simplified answer... 8)

Only dirty pages will be written to swap. That isn't entirely accurate.
Only dirty pages will be written to their backing store, where the backing
store could be, eg, swap, or a mmap()'d file. Clean pages with a backing
store are merely discarded when they're paged out.

-------------------------------------------------------------------
Subscription options: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ/Etiquette: http://www.arm.linux.org.uk/armlinux/mailinglists.php

[Ho Lee]

Thank you. Now everything is clear to me.

-----Original Message-----
From: Russell King - ARM Linux [mailto:***@arm.linux.org.uk]
Sent: Friday, June 06, 2003 1:12 PM
To: Ho Lee
Cc: linux-arm-***@lists.arm.linux.org.uk
Subject: Re: Question about ARM Linux MM

Careful. You missed this:

eor r1, r1, #L_PTE_PRESENT | L_PTE_YOUNG | L_PTE_WRITE |
L_PTE_DIRTY

which inverts the test; we only set HPTE_AP_WRITE if _both_ L_PTE_WRITE
and L_PTE_DIRTY were set. Remember - tst only sets the Z flag when
(op1 & op2) is zero.

What we're doing is ((~op1) & op2) == 0 which is directly equivalent to
(op1 & op2) == op2.

-------------------------------------------------------------------
Subscription options: http://lists.arm.linux.org.uk/mailman/listinfo/linux-arm-kernel
FAQ/Etiquette: http://www.arm.linux.org.uk/armlinux/mailinglists.php