1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305
//! x2APIC Emulation
//!
//! ## Background
//! ### Advanced Programmable Interrupt Controller
//! In interrupt handling, the Advanced Programmable Interrupt Controller (APIC) represents
//! a significantly improved alternative to the conventional Programmable Interrupt Controller (PIC).
//! Originally, the 8235 PIC chip was responsible for interrupt processing, but due to speed and functionality considerations,
//! interrupt handling routine was refined and integrated into the CPU in the cases of the local core interrupts.
//!
//! APIC is implemented as separate Local APIC and IOAPIC components.
//! The former is embedded within each processor, while the latter is located in the system bus.
//! APIC's primary objective is to process interrupts on processors and forward them to designated cores,
//! as well as facilitate Inter-Processor Interrupt (IPI) communication in multicore environments.
//!
//! Over time, Intel has enhanced the APIC to newer versions such as xAPIC and x2APIC.
//! One of the main differences between APIC and xAPIC is their functionality
//! (Increased number of interrupt lines and flexibility in handling different types of interrupts),
//! while x2APIC offers the ability to modify performance behavior through the use of MSR registers.
//!
//! #### LAPIC
//! Each core has a dedicated Local APIC assigned to it, which can generate interrupts and trigger Inter-Processor Interrupts (IPIs)
//! as well as handle up to 0-31 interrupt processing and 32-225 designated interrupt processing.
//! Local APIC, which is directly connected to the processor's cores, performs interrupt-related tasks
//! such as handling I/O devices, APIC Timer, IPI, Performance Monitoring Counter, and Thermal Sensor.
//! LAPIC supports High Precision Event Timer (HPET), which is a high-precision timer that assists each core in obtaining
//! accurate current time values without competing for one timer. If APIC is enabled and HPET is supported, the 8253 PIC is disabled.
//!
//! The definition of how each interrupt behaves can be configured via the Local Vector Table (LVT).
//! LVT is controlled through memory access via MMIO for APIC, xAPIC, and similar devices, and via MSR registers for x2APIC.
//! The Local Vector Table determines which hardware interrupt is forwarded to the core's interrupt pin.
//!
//! #### I/O APIC
//! The I/O APIC is an Interrupt Controller that is responsible for delivering external device interrupts to the appropriate cores.
//! Interrupts delivered through the I/O APIC are forwarded to the Local APIC based on the LVT configuration and ultimately delivered to the core.
//! The I/O APIC is connected to the system bus and is responsible for delivering interrupts generated by hardware or I/O devices to the core.
//!
//! The I/O APIC has a Redirection Table that maps incoming interrupts from each device to a designated interrupt number.
//! Unlike the 8253 interrupt controller, which supports only 16 external interrupts, the APIC supports up to 224 additional interrupts,
//! excluding those assigned to hardware, through redirection.
//!
//! ### APIC Timer - TSC deadline mode
//! APIC Timer modes are separated into three different modes for handling timer interrupts.
//!
//! #### Periodic Mode and One-shot Mode
//! The Periodic Mode is a mode in which software (usually the operating system) requests the
//! APIC to generate periodic timer interrupts by setting a specific time interval.
//! For example, in the Periodic Mode, software requests the APIC to insert a timer interrupt every 1ms.
//!
//! The One-shot Mode is similar to the Periodic Mode but requests the timer interrupt to occur only once.
//! Software requests the APIC to generate a timer interrupt after a certain time interval.
//! For example, software requests the APIC to insert a timer interrupt 1ms later.
//!
//! #### TSC-Deadline Mode
//! The TSC-Deadline Mode has a slightly different characteristic from the other two timer modes.
//! Rather than sending a specific timer interval, this mode requests a timer interrupt when the value of the
//! Time Stamp Counter (TSC) of the core exceeds a specific value.
//! This method uses the TSC timer, which is more accurate than the other two methods that use the CPU frequency,
//! and has a feature that is easy to avoid race conditions.
//! For example, if the current TSC count is 100000, software requests the APIC to generate a timer interrupt when it becomes 100050,
//! which is 100000 + α. Like one-shot mode, software should reprogram the next timer manually to re-trigger the next timer interrupt.
//!
//! Note that, the TSC deadline setting and initiate interrupts are separated.
//! Initiate x2APIC timer interrupts are done by setting Model Specific Register (MSR register) with 0x832 to 0x10 (TSC-Deadline Mode).
//! However, setting a deadline for Local APIC's TSC Deadline Mode is done by setting MSR with 0x6e0 to a timestamp for the next deadline.
//!
//! ## Tasks
//! In this project, you are requested to implement timer interrupt virtualization for guest operating system.
//! You need to implement timer interrupt virtualization for the guest by ensuring that
//! a timer interrupt occurs every tick to schedule threads in the guest operating system.
//!
//! As the guest is unable to use the APIC timer in the host,
//! the host must handle the initialization of the timer and setting of the deadline from the guest.
//! If the TSC value exceeds the guest deadline, a virtual interrupt should be injected into the guest
//! by a separate thread that runs in the hypervisor.
//! This thread is spawned when the guest requests to set the timer as TSC deadline mode and
//! tries to write the timer mode and interrupt vector into the MSR 0x832 (the MSR write is trapped to guest).
//!
//! After the thread is spawned,
//! a deadline setting request via 0x6e0 MSR can be sent to the thread to inject interrupts when the TSC value is more than the set deadline.
//! To share the deadline value between the created thread and the handler for 0x6e0 MSR, the [`channel`] API provided by KeV is used.
//! Injection of the interrupt into the VM should only be done when the VM is not running, as the injected interrupt is handled when VmEntry occurs.
//! To inject the timer interrupt into the running vCPU, the VMM must 1) [`kick`] the vCPU, 2) [`inject`] the interrupt,
//! and then 3) [`resume`] the vCPU to execute the timer interrupt in the guest.
//!
//! [`channel`]: keos::thread::channel::channel
//! [`kick`]: kev::vm::VmOps::kick_vcpu
//! [`inject`]: kev::vcpu::VCpuOps::inject_interrupt
//! [`resume`]: kev::vm::VmOps::resume_vcpu
use alloc::sync::Arc;
use core::arch::x86_64::_rdtsc;
use keos::{
spin_lock::SpinLock,
thread::{
channel::{channel, Sender},
ThreadBuilder,
},
};
use kev::{vcpu::GenericVCpuState, vm::Gpa, Probe, VmError};
use project2::vmexit::msr;
/// X2Apic internal state
pub struct X2ApicInner {
apic_base_0x1b: u64,
tx: Option<Sender<u64>>,
}
/// X2Apic
pub struct X2Apic {
inner: Arc<SpinLock<X2ApicInner>>,
}
const APIC_BASE: u64 = 0xfee0_0800;
impl X2Apic {
pub fn attach(ctl: &mut msr::Controller) {
let inner = Arc::new(SpinLock::new(X2ApicInner {
apic_base_0x1b: APIC_BASE,
tx: None,
}));
// APIC_BASE MSR.
assert!(ctl.insert(
0x1B,
X2Apic {
inner: inner.clone()
}
));
// TP.
assert!(ctl.insert(
0x808,
X2Apic {
inner: inner.clone()
}
));
// eoi
assert!(ctl.insert(
0x80b,
X2Apic {
inner: inner.clone()
}
));
// icr
assert!(ctl.insert(
0x830,
X2Apic {
inner: inner.clone()
}
));
// timer
assert!(ctl.insert(
0x832,
X2Apic {
inner: inner.clone()
}
));
// Susprious interrupt vector.
assert!(ctl.insert(
0x80F,
X2Apic {
inner: inner.clone()
}
));
// lint0
assert!(ctl.insert(
0x835,
X2Apic {
inner: inner.clone()
}
));
// lint1
assert!(ctl.insert(
0x836,
X2Apic {
inner: inner.clone()
}
));
// tsc_deadline
assert!(ctl.insert(
0x6e0,
X2Apic {
inner: inner.clone()
}
));
}
}
impl msr::Msr for X2Apic {
fn rdmsr(
&self,
index: u32,
_p: &dyn Probe,
_generic_vcpu_state: &mut GenericVCpuState,
) -> Result<u64, VmError> {
let inner = self.inner.lock();
match index {
0x1b => Ok(inner.apic_base_0x1b),
0x808 | 0x80b | 0x830 | 0x80f | 0x835 | 0x836 | 0x832 | 0x6e0 => Ok(0),
_ => unreachable!(),
}
}
fn wrmsr(
&mut self,
index: u32,
value: u64,
p: &dyn Probe,
generic_vcpu_state: &mut GenericVCpuState,
) -> Result<(), VmError> {
let mut inner = self.inner.lock();
match index {
0x1b => inner.apic_base_0x1b = value,
// ICR
0x830 => {
let icr = ICR::from_bits_truncate(value as u32);
let (dst, _ipi) = ((value >> 32) as u32, value as u8);
match icr.mode() {
ICRMode::Init => (),
ICRMode::StartUp => {
let entry = unsafe {
(*p.gpa2hva(&generic_vcpu_state.vmcs, Gpa::new(0x467).unwrap())
.unwrap()
.as_mut::<u16>()
.unwrap()
<< 4)
| (*p
.gpa2hva(&generic_vcpu_state.vmcs, Gpa::new(0x469).unwrap())
.unwrap()
.as_mut::<u16>()
.unwrap()
& 0xf)
};
let _ = generic_vcpu_state
.vm
.upgrade()
.unwrap()
.start_vcpu(dst as usize, entry);
}
_ => panic!("Unsupported"),
}
}
0x832 => {
let mode = (value >> 17) & 0b11;
let int = value as u8;
assert_eq!(mode, 0b10, "Only tsc deadline mode is supported.");
let (tx, rx) = channel(1);
todo!();
ThreadBuilder::new("timer_intr").spawn(move || {
// Hint:
// - Receive the deadline from the rx.
// - Wait until time stamp exceeds the deadline.
// - You can get time stamp count with _rdtsc().
// - Kick vcpu and inject the interrupt #int to the vcpu.
// - Resume vcpu.
todo!()
});
inner.tx = Some(tx);
}
0x6e0 => {
todo!()
}
0x808 | 0x80b | 0x80f | 0x835 | 0x836 => (),
_ => unreachable!(),
}
Ok(())
}
}
bitflags::bitflags! {
struct ICR: u32 {
const DEST_1 = 1 << 19;
const DEST_0 = 1 << 18;
const LEVEL = 1 << 15;
const ASSERT = 1 << 14;
const PHYSICAL = 1 << 11;
const MODE_2 = 1 << 10;
const MODE_1 = 1 << 9;
const MODE_0 = 1 << 8;
}
}
impl ICR {
fn mode(&self) -> ICRMode {
match (self.bits() >> 8) & 7 {
0b000 => ICRMode::Fixed,
0b010 => ICRMode::Smi,
0b100 => ICRMode::Nmi,
0b101 => ICRMode::Init,
0b110 => ICRMode::StartUp,
_ => ICRMode::Reserved,
}
}
}
#[derive(Debug)]
enum ICRMode {
Fixed,
Smi,
Nmi,
Init,
StartUp,
Reserved,
}