Infraly, LLC - Notice history

Notice history

Mar 2025

Physgun - General Maintenance – March 26th, 2025
  • Completed
    March 26, 2025 at 4:28 PM
    Completed
    March 26, 2025 at 4:28 PM

    The machine is back online. We will continue to monitor. You should now be able to control your game servers via the control panel. If you have any questions or concerns, reach out to support! :)

  • Update
    March 26, 2025 at 4:26 PM
    Update
    March 26, 2025 at 4:26 PM

    The machine has been upgraded. The system is now booting.

  • In progress
    March 26, 2025 at 4:00 PM
    In progress
    March 26, 2025 at 4:00 PM
    Maintenance is now in progress
  • Update
    March 26, 2025 at 4:00 PM
    Update
    March 26, 2025 at 4:00 PM

    This has been changed to begin at 16:00 UTC to better allow the on-site technician proper time to get prepared.

  • Planned
    March 26, 2025 at 4:00 PM
    Planned
    March 26, 2025 at 4:00 PM

    Dear user,

    Please refer to a very important maintenance announcement that may have an impact on hosting services:

    Physgun - General Maintenance – March 26th, 2025

    Services Affected: CHI-S-GAME-2
    Start Date: March 26th, 2025 @ 16:00 UTC
    Estimated End Date: March 26th, 2025 @ 16:30 UTC

    Summary: Our team will perform upgrades to CHI-S-GAME-2 to further improve performance, reliability, and capability.

    Impact on Customer: Game servers on CHI-S-GAME-2 will be powered down as we will be deracking the machine and upgrading the components. Once upgraded, game servers will be controllable via the game panel. To find out if you are on this machine, please refer to the game panel.

    Updates: We will provide updates via our status page throughout the maintenance. You can subscribe for alerts on our status page (https://status.infraly.co).

    If you have any questions, don't hesitate to contact our client relations team via support ticket. Thank you again for your cooperation and support!

    Sincerely,

    Infraly, LLC DBA Hosturly, Physgun, WISP, & Buildurly
    a: 1636 N Cedar Crest Blvd, #122, Allentown, PA 18104, USA
    e: hello@infraly.co
    p: +1 (833) INF-RALY
    w: infraly.co / hosturly.com / physgun.com / wisp.gg / buildurly.com

WISP – General Maintenance – March 22nd, 2025
  • Completed
    March 22, 2025 at 8:00 PM
    Completed
    March 22, 2025 at 8:00 PM
    Maintenance has completed successfully
  • In progress
    March 22, 2025 at 7:00 PM
    In progress
    March 22, 2025 at 7:00 PM
    Maintenance is now in progress
  • Planned
    March 22, 2025 at 7:00 PM
    Planned
    March 22, 2025 at 7:00 PM

    Dear user,

    Please refer to a very important maintenance announcement that may have an impact on hosting services:

    WISP - General Maintenance – March 22nd, 2025

    Services Affected: WISP Front & Back-end Services
    Start Date: March 22nd, 2025 @ 19:00 UTC
    Estimated End Date: March 22nd, 2025 @ 20:00 UTC

    Summary: Our team will perform changes to the infrastructure of WISP to further improve performance, reliability, and capability.

    Impact on Customer: We do not anticipate disruptions; however, we have scheduled this out of an abundance of caution and transparency. Please note that game servers are not affected - WISP daemons will continue to operate.

    Updates: We will provide updates via our status page throughout the maintenance. You can subscribe for alerts on our status page (https://status.infraly.co).

    If you have any questions, don't hesitate to contact our client relations team via support ticket. Thank you again for your cooperation and support!

    Sincerely,

    Infraly, LLC DBA Hosturly, Physgun, WISP, & Buildurly
    a: 1636 N Cedar Crest Blvd, #122, Allentown, PA 18104, USA
    e: hello@infraly.co
    p: +1 (833) INF-RALY
    w: infraly.co / hosturly.com / physgun.com / wisp.gg / buildurly.com

Infraly - Chicago - Preparation for edge0 Replacement
  • Completed
    March 18, 2025 at 4:21 PM
    Completed
    March 18, 2025 at 4:21 PM

    Our network team is hands-off for the day. No disruption occurred. We will be posting a maintenance post for when we anticipate replacing edge0 completely. :)

  • In progress
    March 18, 2025 at 2:30 PM
    In progress
    March 18, 2025 at 2:30 PM
    Maintenance is now in progress
  • Planned
    March 18, 2025 at 2:30 PM
    Planned
    March 18, 2025 at 2:30 PM

    Dear user,

    Please refer to a very important maintenance announcement that may have an impact on hosting services:

    Preparation for edge0 Replacement – March 18th, 2025

    Services Affected: Chicago, Illinois Data Center
    Start Date: March 18th, 2025 @ 14:30 UTC
    Estimated End Date: March 18th, 2025 @ 18:30 UTC

    Summary: Preparing for the replacement of edge0, one of our routers that was deprioritized from traversing traffic in a previous maintenance post due to slowly dying resulting in infrequent blips.

    Impact on Customer: During this time, our network team will be applying changes to our network core to prepare for the replacement of edge0. Please note that there is an expected blip to occur at least once during this maintenance.

    Updates: We will provide updates via our status page throughout the maintenance. You can subscribe for alerts on our status page (https://status.infraly.co).

    If you have any questions, don't hesitate to contact our client relations team via support ticket. Thank you again for your cooperation and support!

    Sincerely,

    Infraly, LLC DBA Hosturly, Physgun, WISP, & Buildurly
    a: 1636 N Cedar Crest Blvd, #122, Allentown, PA 18104, USA
    e: hello@infraly.co
    p: +1 (833) INF-RALY
    w: infraly.co / hosturly.com / physgun.com / wisp.gg / buildurly.com

Infraly - Global - Path-Routed Services - Packet Loss
  • Resolved
    Resolved

    At 19:48 UTC on March 16th, 2025, we received the all-clear from Path that GTT had completed their maintenance. Since then, there have been no issues with connections. We will continue to monitor as always.

  • Update
    Update

    Path has notified us that one of their upstreams, GTT, is performing emergency maintenance to hopefully correct the issues previously mentioned in this incident post. This maintenance will begin at 15:45 UTC today and is expected to end at 19:45 UTC. Please note that there may be a blip in connectivity for Path-routed services that traverse the PoPs undergoing maintenance.

  • Update
    Update

    Path has notified us that one of their upstreams, GTT, is performing emergency maintenance to hopefully correct the issues previously mentioned in this incident post. This maintenance will begin at 6:00 UTC today and is expected to end at 7:00 UTC. Please note that there may be a blip in connectivity for Path-routed services that traverse the PoPs undergoing maintenance.

  • Identified
    Identified

    We are aware of issues currently with Path-routed services worldwide. We are currently in communication with them as this is outside of our control. Please note that no protection offering is perfect when not controlled by us: Path and Neoprotect both encountered similar issues today.

Feb 2025

Infraly - Chicago - Addressing Frequent Blips
  • Completed
    February 28, 2025 at 11:00 PM
    Completed
    February 28, 2025 at 11:00 PM
    Maintenance has completed successfully
  • Update
    February 28, 2025 at 1:50 AM
    Update
    February 28, 2025 at 1:50 AM

    Maintenance for this has been completed today. We've found the root cause to these Layer 2 issues relating to the router appliance itself. Luckily we have built the network to be redundant, so we will be shifting traffic over to a working router while we work on getting this defective one replaced.

    Again this maintenance will continue to take place at 9:00 AM Central time to avoid dropping players in the peak hours of the day. In the mean time some changes have been made to lessen the effects of these spikes.

    To clarify, we would've had this done today, however the backup router was still in the process of being setup and was paused while on-site was being developed. We needed a few prerequisites completed from our upstream before we could shift any traffic onto it. Which is why we are continuing this tomorrow as it went too far into prime time today to justify dropping traffic.

  • Update
    February 27, 2025 at 8:33 PM
    Update
    February 27, 2025 at 8:33 PM

    We are pushing some changes, you may notice a slight interruption.

  • In progress
    February 27, 2025 at 3:00 PM
    In progress
    February 27, 2025 at 3:00 PM
    Maintenance is now in progress
  • Planned
    February 27, 2025 at 3:00 PM
    Planned
    February 27, 2025 at 3:00 PM

    Our network team has identified the root cause of the frequent short drops that have plagued the network over the past few days. We understand the severity of these drops and have identified the root cause. To permanently resolve these issues, we will be conducting emergency maintenance at 9:00 AM Central Time. During this maintenance, you will experience brief disruptions as mass changes will be done. We apologize in advance for any inconvenience this may pose to you, especially over these past few weeks where we have suffered instability from one of our upstream providers as well as the network maintenance we conducted to upgrade our core late last month. To be frank, the timing could not have been worse.

    For transparency, the issue stems from a Layer 2 issue inside our EVPN-VXLAN network, which is affecting our spine switches that process essentially all traffic on our network. As a result, this was felt widespread and not just related to one upstream or one cabinet. This issue appears to have sprung from the preparation of onboarding our on-prem protection suite into our core. We have been working at a fast pace to get that online and operational in hopes of shifting all traffic over in order to provide stability.

    We know the past few weeks have been difficult, and we recognize the importance of a reliable network for our customers and are tirelessly working to improve our offerings.

Infraly - Chicago - More Progress With On-Site Protection
  • Completed
    March 01, 2025 at 3:00 PM
    Completed
    March 01, 2025 at 3:00 PM
    Maintenance has completed successfully
  • Update
    February 25, 2025 at 10:44 PM
    Update
    February 25, 2025 at 10:44 PM

    We have completed maintenance for today, this extension saw no external loss to production switches. We have made a lot of real, meaningful progress today. We are around the home stretch now, simply a few things to tidy up before we push more live traffic to onsite!

    We will be continuing our work again tomorrow at 9:00 AM Central to avoid any potential interruptions during peak hours.

  • Update
    February 25, 2025 at 7:36 PM
    Update
    February 25, 2025 at 7:36 PM

    Hello everyone, apologies for the short notice but we will be extending the previous maintenance for a few more hours.

    Why? Because today we’ve made some crazy good progress, and we think onsite can realistically be done very soon. We everything is online we even have some live traffic being pushed to it.

    To iterate again, this will be the solution for us to attempt to seamlessly move traffic from the global upstream to our localized mitigation stack when they have issues. Effectively keeping uptime with as little to no drops as possible.Thank you for bearing with us, we will send out some cool pictures soon

  • In progress
    February 25, 2025 at 3:00 PM
    In progress
    February 25, 2025 at 3:00 PM
    Maintenance is now in progress
  • Planned
    February 25, 2025 at 3:00 PM
    Planned
    February 25, 2025 at 3:00 PM

    At specifically 9AM Central Time we will be continuing development with our onsite network. We apologize for any interruptions this may cause, we are doing this early in the AM to ensure we impact as little traffic as possible.

    We understand the frequent issues from upstream isn't making this an easy pill to swallow, but we must continue work on this to ensure we are independent from future network issues once Onsite goes live.

    To give context on what onsite will bring for those who may have missed it:

    Infraly Onsite DDoS Protection will allow us to have a localized Path environment that will be isolated from the global network. This will allow us to attempt to seamlessly, without dropping traffic move between the global network and local network when needed.

    Take for example the issues that we have been facing now, these are all from the global network with Onsite we would be-able to push traffic onto our local mitigation stack, effectively isolating us from global network issues, ensuring full uptime. This also works in reverse, if the local network was to go offline for some reason, we would route back onto the global network, while again attempting to not drop any traffic.

    This system is extremely complex, and has been under development for a while now, we are speeding up progress but also doing so at the least impactful times, we understand no downtime is ideal, however this must be done in order to ensure we have a fully redundant network. I am sure everyone is tired of these upstream issues, we are too, and we truly believe onsite is the solution we are all looking for.

Physgun – Emergency Maintenance on LON-S-GAME-1 – February 23rd, 2025
  • Completed
    February 23, 2025 at 9:35 PM
    Completed
    February 23, 2025 at 9:35 PM

    This has been completed successfully. All servers on LON-GAME-1 should be back up and running. We sincerely apologize for the inconvenience this may have caused.

    As mentioned previously, we will be providing SLA credit to all clients affected. Please open a ticket on the billing site if your game server was affected by this. We will verify and then credit accordingly.

  • In progress
    February 23, 2025 at 6:50 PM
    In progress
    February 23, 2025 at 6:50 PM
    Maintenance is now in progress
  • Planned
    February 23, 2025 at 6:50 PM
    Planned
    February 23, 2025 at 6:50 PM

    Dear user,

    Please refer to a very important maintenance announcement that may have an impact on hosting services:

    Emergency Maintenance on LON-S-GAME-1 – February 23rd, 2025

    Services Affected: Physgun's LON-S-GAME-1
    Start Date: February 23rd, 2025 @ 18:50 UTC
    Estimated End Date: February 23rd, 2025 @ 22:50 UTC

    Summary: We have identified a critical issue on LON-S-GAME-1 and need to perform updates and migrations to ensure stability for the machine.

    Impact on Customer: During the maintenance, users on LON-S-GAME-1 will experience a service disruption. Due to the short notice of this maintenance, we are offering SLA credit for this. More details will come for this later.

    Updates: We will provide updates via our status page throughout the maintenance. You can subscribe for alerts on our status page (https://status.infraly.co).

    If you have any questions, don't hesitate to contact our client relations team via support ticket. Thank you again for your cooperation and support!

    Sincerely,

    Infraly, LLC DBA Hosturly, Physgun, WISP, & Buildurly
    a: 1636 N Cedar Crest Blvd, #122, Allentown, PA 18104, USA
    e: hello@infraly.co
    p: +1 (833) INF-RALY
    w: infraly.co / hosturly.com / physgun.com / wisp.gg / buildurly.com

Jan 2025

WISP – Emergency General Maintenance – January 29th, 2025
  • Completed
    January 29, 2025 at 8:51 PM
    Completed
    January 29, 2025 at 8:51 PM

    The maintenance has been completed with under 5 minutes of downtime during the maintenance period.

  • In progress
    January 29, 2025 at 8:00 PM
    In progress
    January 29, 2025 at 8:00 PM
    Maintenance is now in progress
  • Planned
    January 29, 2025 at 8:00 PM
    Planned
    January 29, 2025 at 8:00 PM

    Dear user,

    Please refer to a very important maintenance announcement that may have an impact on hosting services:

    Emergency General Maintenance – January 29th, 2025

    Services Affected: WISP Front-end Services
    Start Date: January 29th, 2025 @ 3:00 PM ET
    Estimated End Date: January 29th, 2025 @ 4:00 PM ET

    Summary: Our team will perform emergency maintenance to prepare to implement the new US infrastructure later this week.

    Impact on Customer: Front-end services will become inaccessible for a short period in this maintenance. We have scheduled an hour window for caution, but do not anticipate a loss of accessibility for the entire window. Please note that game servers are not affected - WISP daemons will continue to operate.

    Updates: We will provide updates via our status page throughout the maintenance. You can subscribe for alerts on our status page (https://status.infraly.co).

    If you have any questions, don't hesitate to contact our client relations team via support ticket. Thank you again for your cooperation and support!

    Sincerely,

    Infraly, LLC DBA Hosturly, Physgun, WISP, & Buildurly
    a: 1636 N Cedar Crest Blvd, #122, Allentown, PA 18104, USA
    e: hello@infraly.co
    p: +1 (833) INF-RALY
    w: infraly.co / hosturly.com / physgun.com / wisp.gg / buildurly.com

[EXTENDED] Infraly – Emergency Network Maintenance – January 22nd - 29th, 2025
  • Completed
    January 30, 2025 at 5:44 AM
    Completed
    January 30, 2025 at 5:44 AM

    We are now exiting our maintenance period, and we sincerely apologize for the extended duration and any disruptions this may have caused. We understand that any downtime can cause frustration, and we deeply regret the impact on your operations.

    This overhaul of our core infrastructure was necessary to enhance performance, reliability, and scalability, ensuring a stronger network moving forward. We truly appreciate your patience and support.

    Further details on the enhancements will be shared soon.

  • Update
    January 29, 2025 at 6:57 PM
    Update
    January 29, 2025 at 6:57 PM

    Over the next 10 minutes, a blip will disrupt some traffic routed over a core network device. We apologize for the inconvenience. Our maintenance is one step closer to being completed after this. :)

  • Update
    January 29, 2025 at 6:05 PM
    Update
    January 29, 2025 at 6:05 PM

    Over the next 20 minutes, there may be a few blips that will disrupt some traffic routed over a core network device. We apologize for the inconvenience. Our maintenance is one step closer to being completed after this. :)

  • Update
    January 29, 2025 at 4:41 PM
    Update
    January 29, 2025 at 4:41 PM

    Over the next 60 minutes, there may be a few blips that will disrupt some traffic routed over a core network device. We apologize for the inconvenience. Our maintenance is one step closer to being completed after this. :)

  • Update
    January 29, 2025 at 1:45 AM
    Update
    January 29, 2025 at 1:45 AM

    Over the next 20 minutes, there may be a few blips that will disrupt some traffic routed over a core network device. We apologize for the inconvenience. Our maintenance is one step closer to being completed after this. :)

  • Update
    January 29, 2025 at 1:14 AM
    Update
    January 29, 2025 at 1:14 AM

    Over the next 20 minutes, there may be a few blips that will disrupt some traffic routed over a core network device. We apologize for the inconvenience. Our maintenance is one step closer to being completed after this. :)

  • Update
    January 28, 2025 at 11:24 PM
    Update
    January 28, 2025 at 11:24 PM

    Over the next 60 minutes, there may be a few blips that will disrupt some traffic routed over a core network device. We apologize for the inconvenience. Our maintenance is one step closer to being completed after this. :)

  • Update
    January 28, 2025 at 2:22 AM
    Update
    January 28, 2025 at 2:22 AM

    Our network engineering team is finished for the day and will regroup tomorrow. Our goal is to finish this maintenance period tomorrow before the Wednesday deadline.

    Tomorrow, Tuesday, January 28th, 2025, we anticipate possible disruptions as we move traffic. We will provide updates whenever any disruption of service is expected. We strongly encourage you to subscribe to this maintenance post to receive real-time notifications.

    We sincerely apologize for the inconvenience caused by this maintenance period. Once this maintenance is completed, we will be releasing an announcement regarding the upgrades completed and how they will lead to a more reliable service offering.

  • Update
    January 27, 2025 at 5:43 AM
    Update
    January 27, 2025 at 5:43 AM

    Please note that this maintenance period has been extended to Wednesday, January 29th, 2025 11:59 pm CT. This extension is necessary to ensure the successful implementation of critical upgrades.

    Our network engineering team is finished for the night and will regroup tomorrow morning to begin additional work.

    We will provide updates whenever any disruption of service is expected. We strongly encourage you to subscribe to this maintenance post to receive real-time notifications.

    We sincerely apologize for the inconvenience caused by this maintenance period. Once this maintenance is completed, we will be releasing an announcement regarding the upgrades completed and how they will lead to a more reliable service offering.

  • Update
    January 27, 2025 at 3:27 AM
    Update
    January 27, 2025 at 3:27 AM

    Over the next 20 minutes, a blip will disrupt some traffic routed over a core network device. We apologize for the inconvenience. Our maintenance is one step closer to being completed after this. :)

  • Update
    January 27, 2025 at 1:42 AM
    Update
    January 27, 2025 at 1:42 AM

    Within the next 20 minutes, a blip will disrupt some traffic routed over a core network device. We apologize for the inconvenience. Our maintenance is one step closer to being completed after this. :)

  • Update
    January 26, 2025 at 9:57 PM
    Update
    January 26, 2025 at 9:57 PM

    At 4:05 pm CT, roughly 10 minutes from now, a blip will disrupt some traffic routed over an edge network device. We apologize for the inconvenience. Our maintenance is one step closer to being completed after this. :)

  • In progress
    January 23, 2025 at 1:00 AM
    In progress
    January 23, 2025 at 1:00 AM
    Maintenance is now in progress
  • Planned
    January 23, 2025 at 1:00 AM
    Planned
    January 23, 2025 at 1:00 AM

    Dear user,

    Please refer to a very important maintenance announcement that may have an impact on hosting services:

    Infraly – Emergency Network Maintenance – January 22nd - 26th, 2025
    Location Affected: Chicago, Illinois
    Start Date: January 22nd, 2025
    Estimated End Date: January 26th, 2025

    Summary: We will be performing maintenance on our core network infrastructure to further improve performance, reliability, and capability.

    Impact on Customer: We will be limiting the impact on customers by isolating equipment that is being replaced while we upgrade our core. During this time, redundant equipment will remain online. Rack colocation clients with a single connection will be subject to at least one brief disruption as those cross-connects are moved to the upgraded equipment. While we have extensively planned this maintenance behind the scenes to limit any impact, please note that there is always the possibility for a brief disruption of service throughout maintenance periods. All possible measures are being taken to minimize disruptions.

    Updates: We will provide updates via our status page throughout the maintenance. You can subscribe for alerts on our status page (https://status.infraly.co).

    If you have any questions, don't hesitate to contact our client relations team via support ticket. Thank you again for your cooperation and support!

    Sincerely,

    Infraly, LLC DBA Hosturly, Physgun, WISP, & Buildurly
    a: 1636 N Cedar Crest Blvd, #122, Allentown, PA 18104, USA
    e: hello@infraly.co
    p: +1 (833) INF-RALY
    w: infraly.co / hosturly.com / physgun.com / wisp.gg / buildurly.com

Jan 2025 to Mar 2025

Next