Re: apologies
We deployed the AO3 this weekend.
Since then![[personal profile]](https://www.dreamwidth.org/img/silk/identity/user.png)
lim has apologized and resigned. I don't believe she has anything to apologize for, but I don't blame her for resigning. It's demoralizing to see your hard work torn to shreds by people who have no idea of the constraints we work under. Not just the constraints of time and personnel, but the constraints of more than fifty different parsers and many complex accessibility requirements. And nobody wants to work somewhere where their work will be hated so much that people will rudely and publicly vilify it.
Hele has apologized on behalf of the board and promised to make sure it never happens again. I don't think the board has anything to apologize for either, unless they were the cause of the vitriol, which as far as I know is not the case. But I do think their promise is unrealistic. No matter what they do, they'll never be able to make sure it never happens again. In twenty-five years I have not once seen a deploy in RL which was bug free and didn't cause a single complaint from the users. And that's when there are full-time highly paid developers, highly paid release engineers, and very highly paid project managers. All of whom have stock options which will be worth nothing unless they make a success of the company and so are extremely vested in the outcome. No matter how much we improve the process or the code, we will never be able to produce a 100% bug-free deploy that everyone loves. It's not possible. Honest, it's really just not possible.
But if you're nice, you apologize when people complain.
Well, I do not apologize for this deploy.
Who am I?
I'm the co-chair of OTW's System committee.
I'm a senior coder for the archive and one of the owner's of the archive code on github. (ambtus).
In RL I'm a senior unix system administrator. If you're interested in my professional qualifications, check out my public Linkedin profile. I'm not a professional developer, but I have twenty-five years experience working with or in development departments. First in QA, and then as a sysadmin where I work closely with development and release engineering.
I wrote the code that pushes the code to production. I made the changes on the server which were necessary for the new code to be deployed. I control access to the servers for the people who push the code. I trained and encouraged and enabled the person who did the push this time.
My actual quote before the deploy was:
If anyone is responsible for this code making it onto production, it is me.
And I do not apologize for this deploy.
This deploy, despite receiving an extraordinary amount of negative feedback (and an unusually large amount of cruel and hurtful feedback in the form of personal attacks against the developers), was not more buggy than usual, or worse than usual, or more than typically bad.
There have been other deploys that melted the servers. Deploys that had huge security bugs. Deploys which broke major functionality. Deploys which were a great deal worse than this one. I worked for twenty-four hours straight last time to recover from the last deploy because we did something to the code which melted the database server.
That's not to say the current code should be lauded just because it's better than what we had before. Not at all. I know more than anyone just how much work we still have to do. Believe me, we know what the problems are. We're not ignoring the support requests or the tweets or the LJ posts. We don't think our process is perfect. For that matter, I don't even think our process is very good but we're continually working to improve it.
But we don't have anything to apologize for. We're doing our best under very difficult circumstances. Developers are not crawling out of the woodwork begging to be allowed to work on this project. We don't have release engineers and project managers waiting in the wings to help us. But the users are lining up in droves to join and usage just continues to increase by leaps and bounds.
None of the people who worked on this release have anything to apologize for. The board does not need to apologize for not having stepped in to prevent it from happening or promise it will never happen again.
So.
If you don't like something we do, please open a support ticket. If you're nice about it we'll do our best to fix it as soon as we can.
Actually, we'll do our best to fix it even if you're rude and hateful, because our support volunteers are a lot nicer than I am and will pass on your issues (although not your vitriol) to the developers no matter what.
Or fix it yourself. We put the code on github so that anyone can fork it and send us a pull request. We'll be more than grateful for anything we can use.
But don't ask me to apologize for what we've produced so far. Because it's an amazing and complex piece of work, completely coded from scratch by amateurs and volunteers.
And we have nothing at all to apologize for.
Since then
lim has apologized and resigned. I don't believe she has anything to apologize for, but I don't blame her for resigning. It's demoralizing to see your hard work torn to shreds by people who have no idea of the constraints we work under. Not just the constraints of time and personnel, but the constraints of more than fifty different parsers and many complex accessibility requirements. And nobody wants to work somewhere where their work will be hated so much that people will rudely and publicly vilify it.Hele has apologized on behalf of the board and promised to make sure it never happens again. I don't think the board has anything to apologize for either, unless they were the cause of the vitriol, which as far as I know is not the case. But I do think their promise is unrealistic. No matter what they do, they'll never be able to make sure it never happens again. In twenty-five years I have not once seen a deploy in RL which was bug free and didn't cause a single complaint from the users. And that's when there are full-time highly paid developers, highly paid release engineers, and very highly paid project managers. All of whom have stock options which will be worth nothing unless they make a success of the company and so are extremely vested in the outcome. No matter how much we improve the process or the code, we will never be able to produce a 100% bug-free deploy that everyone loves. It's not possible. Honest, it's really just not possible.
But if you're nice, you apologize when people complain.
Well, I do not apologize for this deploy.
Who am I?
I'm the co-chair of OTW's System committee.
I'm a senior coder for the archive and one of the owner's of the archive code on github. (ambtus).
In RL I'm a senior unix system administrator. If you're interested in my professional qualifications, check out my public Linkedin profile. I'm not a professional developer, but I have twenty-five years experience working with or in development departments. First in QA, and then as a sysadmin where I work closely with development and release engineering.
I wrote the code that pushes the code to production. I made the changes on the server which were necessary for the new code to be deployed. I control access to the servers for the people who push the code. I trained and encouraged and enabled the person who did the push this time.
My actual quote before the deploy was:
i'm shaking in my boots, because last time was such a disaster, but i guess we're as ready as we ever will be
If anyone is responsible for this code making it onto production, it is me.
And I do not apologize for this deploy.
This deploy, despite receiving an extraordinary amount of negative feedback (and an unusually large amount of cruel and hurtful feedback in the form of personal attacks against the developers), was not more buggy than usual, or worse than usual, or more than typically bad.
There have been other deploys that melted the servers. Deploys that had huge security bugs. Deploys which broke major functionality. Deploys which were a great deal worse than this one. I worked for twenty-four hours straight last time to recover from the last deploy because we did something to the code which melted the database server.
That's not to say the current code should be lauded just because it's better than what we had before. Not at all. I know more than anyone just how much work we still have to do. Believe me, we know what the problems are. We're not ignoring the support requests or the tweets or the LJ posts. We don't think our process is perfect. For that matter, I don't even think our process is very good but we're continually working to improve it.
But we don't have anything to apologize for. We're doing our best under very difficult circumstances. Developers are not crawling out of the woodwork begging to be allowed to work on this project. We don't have release engineers and project managers waiting in the wings to help us. But the users are lining up in droves to join and usage just continues to increase by leaps and bounds.
None of the people who worked on this release have anything to apologize for. The board does not need to apologize for not having stepped in to prevent it from happening or promise it will never happen again.
So.
If you don't like something we do, please open a support ticket. If you're nice about it we'll do our best to fix it as soon as we can.
Actually, we'll do our best to fix it even if you're rude and hateful, because our support volunteers are a lot nicer than I am and will pass on your issues (although not your vitriol) to the developers no matter what.
Or fix it yourself. We put the code on github so that anyone can fork it and send us a pull request. We'll be more than grateful for anything we can use.
But don't ask me to apologize for what we've produced so far. Because it's an amazing and complex piece of work, completely coded from scratch by amateurs and volunteers.
And we have nothing at all to apologize for.
no subject
no subject
I haven't seen anyone asking for an apology either. This post was in response to the people who did apologize, not in response to anyone who might have asked for one.
I have seen a lot of requests for increased clarity and comprehensiveness
Skud's post is a wealth of useful constructive criticism and we'll get around to implementing as much of it as we can as soon as we can within the limitations and constraints that we work under. We set up the in-house volunteer system to help train people who don't know how to code. People who already know what they're doing can fork the code and submit pull requests. Which skud has already done. It's one of the main reasons we moved to github.
Is the AO3 code documented anywhere?
The AO3 code is documented in the code. It's not documented very well. If you don't want to wait for us to fix it and would like to fork it and improve its documentation yourself, please, feel free.
no subject
I'm not currently capable of writing such documentation due to time constraints, but I do think, as someone who's interested in doing driveby coding, that the more you guys attempt to recruit casual, experienced coders, the better off you'll be. I'd be happy to talk with someone about working on documentation in the future, though, if I knew there was a place to put it, and what the plans for community documentation were/are. :)
(sorry about the borked reply - that's what I get for making comments during finals week)
no subject
no subject