Alert on Homa’s Performance in PowerTCP Paper

We were informed that the Homa NS3 simulator (developed by a third-party) that portrays itself as the official Homa simulator is prone to bugs and doesn’t fully implement what John Ousterhout considers as true specification of Homa. After a long discussion with John, we agreed that the performance of Homa presented in PowerTCP is possibly prone to Homa’s NS3 simulator issues. As a result, there are still unresolved anomalies with the Homa simulations, so the paper’s conclusions only relating to Homa cannot be relied on until the similator issues are resolved.

We hence request the networking community to pay extra attention in making claims on Homa based on the NS3 version of Homa simulator. As a side note, we notice that the OMNET++ version of Homa simulator also doesn’t fully implement Homa. We suggest to consult John if you wish to evaluate Homa.

Note: At the time we selected the simulator to use for Homa, it stated “This implementation is created in guidance of the protocol creators and maintained as the official ns-3 implementation of the protocol” and reproduced the original Homa paper’s results.

The official simulator of PowerTCP is available online, functional and reproduces our results. Importantly, it includes various practical modules of a datacenter setup: state-of-the-art congestion control algorithms as well as buffer management algorithms. Please feel free to use our simulator with high confidence. We are also actively responding via github on any issues raised. To that end, we encourage you to use our simulator either to evaluate PowerTCP or for any further protocol developements.