Update Secure Timer Registers with OPTEE

[SimonWan]

Hi experts @etienne-lms @igoropaniuk ,

Two months before I asked one question about running secure-only process without involving the normal world activities. In that post, I got a possible solution which is configuring the secure timer registers to raise secure interrupt for world switching.

However, after many times tests (I tried to write these registers in S-EL1 static TA and EL3 BL31 / OPTEE-SPD), I still cannot make the secure timer works correctly.

Here are some results I got:

1. When I configure the registers CNTPS_CVAL_EL1 and CNTPS_CTL_EL1 in BL31, I can get an interrupt raised during the booting time and the console prints Disabling unhandled interrupt 29. By getting this result, I assume my writing to these registers works correctly;
2. After booting time, I check the registers again with my sta, all value go back to the 0;
3. I add some code in the TEESMC_OPTEED_RETURN_FIQ_DONE case of opteed_smc_handler to update the registers;
4. I manually input in my console to activate the interrupt 116 so my code can run when the OPTEE OS issues SMC RETURN. During one smc return, I print the registers at first:

console_itr_cb: cpu 4: got 0xd 
Generic timer counter CNTPCT_EL0: 3167e1487
Secure timer compare value CNTPS_CVAL_EL1: 0
Secure timer control value CNTPS_CTL_EL1: 0

Then I add code for updating timer registers. After writing, I read the registers again:

Generic timer counter CNTPCT_EL0: 31687bd42.
Secure timer compare value CNTPS_CVAL_EL1: 31804f28e.
Secure timer control value CNTPS_CTL_EL1: 1

As you can see, at the EL3 level it looks like I updated the registers correctly. However, once it's done and I generate the FIQ again, all the registers go back to the default 0 value.

I really don't have any idea about what the problem might be so I decided to put it out here and see if anyone else could help me out.

Thank you in advance. Any suggestion is welcome!

Bests,
Simon

P.S.
Here is my assembly code for udpating these registers:

readtcontrol: 
	ISB
	MRS	X0, CNTPS_CTL_EL1
	RET

writetcontrol: 
	MSR	CNTPS_CTL_EL1, X0
	ISB
	RET

readcomparer: 
	ISB
	MRS	X0, CNTPS_CVAL_EL1
	RET

writecomparer: 
	MSR	CNTPS_CVAL_EL1, X0
	ISB
	RET

readcounter: 
	ISB
	MRS	X0, CNTPCT_EL0
	RET

I define these functions as global functions and call them in the code I mentioned above.

[igoropaniuk]

Hi @SimonWan,

If I understand you correctly you have to re-enable (setting CNTPS_CTL_EL1) the timer each time in your secure interrupt handler.

There is a good example of handling the secure timer interrupt in the Test Secure Payload Dispatcher (TSPD) service in ARM trusted firmware.
(check paragraph about it's usage in https://github.com/ARM-software/arm-trusted-firmware/blob/master/docs/interrupt-framework-design.rst#test-secure-payload-behavior)

Implementation of handler is here https://github.com/ARM-software/arm-trusted-firmware/blob/master/bl32/tsp/tsp_timer.c#L42, pay attention on:

	/*
	 * Disable the timer and reprogram it. The barriers ensure that there is
	 * no reordering of instructions around the reprogramming code.
	 */
	isb();
	write_cntps_ctl_el1(0);
	tsp_generic_timer_start();
	isb();

Hope it helps
BR,
Igor

[SimonWan]

Hi @igoropaniuk ,

Actually I have tried to re-enable the CNTPS_CTL_EL1 in the handler.
For that version, after I call the function similar to tsp_generic_timer_start(), I check the register with value 1. However, when the interrupt handler is done and returns to the normal world, I switch back to the secure world to read the value and the register goes back to 0.

In this case, I'm thinking maybe my write is lost or re-write by some part of the OPTEE-SPD but I can't find any related document to solve the issue.

Thank you for the reply.

[igoropaniuk]

@SimonWan,

1. However, when the interrupt handler is done and returns to the normal world, I switch back to the secure world to read the value and the register goes back to 0.

I'm not sure I understand what actually you mean by "the interrupt handler is done and return to the normal world and switch back" (you are checking CNTPS_CTL_EL1 in your interrupt handler or somewhere else).
Could you please show examples of you code of the interrupt handler and how you register it?

BTW, did you receive in SW output error messages like "unhandled interrupt" etc:
Disabling unhandled interrupt...
or
Disabling interrupt %zu not handled by handler...

BR,
Igor

[SimonWan]

@igoropaniuk , I add the code for updating CNTPS_CTL_EL1 and other registers in three places:

1. the BL31;
2. I have a STA for setting the registers;
   3 I add the code at the TEESMC_OPTEED_RETURN_FIQ_DONE case of opteed_smc_handler to make sure after OPTEE handles the interrupt, the reigsters can be set as the correct value.
   My code is just reusing the functions of tsp_timer.c.

For the SW output error question, I did see the message during booting time after BL31. I guess this means my configuration at BL31 works.
However, after both worlds finish the initialization, all register rolls back to 0.
If I call the STA for writing these registers, I can trigger the interrupt one time and the secure world console will print the SW output error message.
The problem is, if I set the CNTPS_CVAL_EL1 with a value larger than the current time and the core switch back to the normal world, then the secure timer interrupt never comes up again.

Now I have changed my SP/SPD to the TSP of arm-tf and the secure timer works.
My simple guess is the context switch of OPTEE-SPD somehow re-writes these registers but I can't solve it now.