The size of the driver does not make it fast or slow. In this case, you have low delay because its a sealed box. The HS-24 would perfectly fine for music, but you do need to couple it with a large box because of its higher Q of about 0.5. A big sealed box would be perfect for this driver.
Also, to understand why group delay is a much more apparent with a port / horn (compared to just a driver in a sealed box) because the port, unlike the driver acts like a spring mass system. When you apply a voltage to the driver, its ridge and starts to accelerate more or less as a rigid body. But when yo