Differences

This shows you the differences between two versions of the page.

--- sumjulyevent [2025/07/09 12:27] – created astroadmin
+++ sumjulyevent [2025/07/09 14:37] (current) – astroadmin
@@ Line 1: / Line 1: @@
 ====== Report of July 5th, 2025 events resulting in critical failure ======
+=== General Timeline ===
+  - At ~1am July 5th, 2025 SSP TA Yujean reached out to Zach concerning and issue with the dome and weather station not talking / connecting to ACP. After restarting TMO's desktop and turning on/off the USB port controlling both of these devices, Zach was able to remotely recover. Everything was working and connected at the start of the night when Zach did their usually pre-SSP use checks. Zach monitored the entire shift to ensure the connection held. Zach instructed the SSP team to *not* start the Scheduler. There were no other issues through the rest of the shift.
+  - At 3am July 5th, 2025 the SSP accidently started the scheduler despite prior instructions. Zach took over at this point to wait for a safe place to stop the scheduler.
+  - At 3:15 July 5th, 2025 Zach closed the dome. As the dome was closing, it stopped ~mid way, prompted a "SHUTTER ERROR" in ACP, and using TMOCAM a wire was visibly hanging down from the pulley system. The weather for the rest of the night was supposed to be fine, so Zach left it as is. Zach called the NMSU after hours desk. Since weather outlook was only ~1% for the weekend it was left till Monday.
+  - At ~8am July 5th, 2025 Zach checked in on the telescope remotely using TMOCAM it was noticed that the shutters have moved. As viewed from TMOCAM, the left shutter was in a "closed" position, but was crooked (bottom was more closed than the top). The right shutter was fully opened. Zach went up to TMO to physically inspect. The chain was completely wrapped around the motor gear. Zach called the afterhours Work Order desk again to follow up and update to this new concerning situation.
+  - At ~4pm July 6th, Zach noticed that the local weather's rain chance went from ~1% to 40-60% chance. With the dome in an opened / critical state, Zach hiked up to TMO with tarps and cords to cover the telescope to mitigate any potential chance of weather related damage. Chance of rain subsided around ~8pm and Zach left the telescope covered with tarps.
+  - At ~9am July 7th, NMSU F&S assisted Zach in recovering the dome. In doing so we noticed three things: 1) F&S expressed that the chain was warped, 2) one of the pulley wheels came off the dome (this was fixed), and 3) the trigger switch that determines the open position snapped off (this wheel is currently on the white table in the "living room" labeled as such).
+=== Ongoing Issues ===
+The limit triggers that define open and close being compromised is one of the main issues we currently face. Additionally, the tension in the pulley system is not correct, causing slack when the dome is opened and closed (Slack is removed over time as the dome sits). Likely, as a result of the incorrect tension in the system, the dome does not have smooth motion when closing (but does when it is opening). Barings at the top need to be lubricated.
+=== Current State ===
+Zach has removed the battery connection from the shutter, thus rendering the shutter incapable of opening. Zach was also able to slightly adjust the cables to allow the dome's ramp to be the cause of the trigger to the closed trigger switch. **HOWEVER** there is still concerns about the tension of the cable.