[Humanist] 27.387 peer review

Humanist Discussion Group willard.mccarty at mccarty.org.uk
Mon Sep 30 07:38:44 CEST 2013


                 Humanist Discussion Group, Vol. 27, No. 387.
            Department of Digital Humanities, King's College London
                       www.digitalhumanities.org/humanist
                Submit to: humanist at lists.digitalhumanities.org



        Date: Sun, 29 Sep 2013 18:46:30 +0100
        From: "James O'Sullivan" <josullivan.c at gmail.com>
        Subject: Re:  27.381 peer review
        In-Reply-To: <20130927034451.B8E262DAB at digitalhumanities.org>


Thanks to all for the informed responses.

I think Dr Terras raises a good point when she points out that the DH
community has changed a lot since 2005 - perhaps the time has come to
revisit the review process in respect of such changes?

There are leaders in this field who, as many have pointed out, are
identifiable as a result of their research interests etc, but there is an
increasingly wider group of scholars who are not so easily identifiable,
and to my mind, the fairest way to appraise their work is either via a
double-blind or open process. I think when a review process is
single-blind, there will always be doubt in relation to selections, and I
think that this is unfair on both reviewers and the reviewed.

There might of course be other logistical reasons to be taken into account;
again, more experienced individuals might be in a position to highlight
these, but I do think that this process should be reconsidered, or at least
opened up to a wider debate.

Just my two cents - and thanks again to all for input.

Best regards,
James

On Fri, Sep 27, 2013 at 4:44 AM, Humanist Discussion Group <
willard.mccarty at mccarty.org.uk> wrote:

>
>                  Humanist Discussion Group, Vol. 27, No. 381.
>             Department of Digital Humanities, King's College London
>                        www.digitalhumanities.org/humanist
>                 Submit to: humanist at lists.digitalhumanities.org
>
>   [1]   From:    "Sara A. Schmidt" <saschmidt8 at gmail.com>
>  (61)
>         Subject: Re:  27.380 peer review?
>
>   [2]   From:    Martin Holmes <mholmes at uvic.ca>
>   (30)
>         Subject: Re: [Humanist] 27.380 peer review?
>
>   [3]   From:    Melissa Terras <melissaterras at gmail.com>
>  (84)
>         Subject: Re:  27.380 peer review?
>
>
>
> --[1]------------------------------------------------------------------------
>         Date: Thu, 26 Sep 2013 03:40:38 -0500
>         From: "Sara A. Schmidt" <saschmidt8 at gmail.com>
>         Subject: Re:  27.380 peer review?
>         In-Reply-To: <20130926061104.D456D3094 at digitalhumanities.org>
>
>
> Ray Siemens would be better able to speak to this issue than me, but I
> think single blind review has been used since ADHO started using ConfTool
> because people were routinely listing their names and affiliations in the
> files they were uploading into conftool so it would have been necessary for
> the PC chair to download each uploaded file in order to make sure no names
> or affiliations were included. I believe individuals submitting panel
> abstracts have historically tended to mention the names and affiliations of
> the various speakers planning to take part in the panel sessions. In
> addition I believe that it was believed that some long term members of the
> digital humanities community were familiar enough with the research and
> writing styles of some of their colleagues so that they would be able to
> make an educated guess as to the authors of a number of the abstracts they
> were being asked to review.
>
> Sara Schmidt
>
> On Thu, Sep 26, 2013 at 1:11 AM, Humanist Discussion Group <
> willard.mccarty at mccarty.org.uk> wrote:
>
> >                  Humanist Discussion Group, Vol. 27, No. 380.
> >             Department of Digital Humanities, King's College London
> >                        www.digitalhumanities.org/humanist
> >                 Submit to: humanist at lists.digitalhumanities.org
> >
> >
> >
> >         Date: Wed, 25 Sep 2013 16:19:49 +0100
> >         From: "James O'Sullivan" <josullivan.c at gmail.com>
> >         Subject: Peer Review Processes
> >
> >
> > Hello all,
> >
> > I was hoping to start a discussion on peer review processes. Looking at
> the
> > submission guidelines for the DH2014, I noticed that the ADHO "has
> > discontinued the anonymization of papers during the review process
> (moving
> > to single-blind review)". I'm wondering why this is, and what people
> think
> > of such a move?
> >
> > I've always been of the opinion that single-blind review has few benefits
> > when compared with double-blind processes, and I've also heard some
> > convincing arguments in relation to more open models.
> >
> > I have limited experience as a reviewer, so I am curious to know if there
> > are any merits to a single-blind review process?
> >
> > Thanks as always...
> > James
> >
> > --
> > *James O'Sullivan *
> > @jamescosullivan  http://twitter.com/jamescosullivan **
> > Web: josullivan.org
> >
> > Twitter: http://twitter.com/jamescosullivan
> > http://twitter.com/#%21/jamescosullivan
> > LinkedIn: http://linkedin.com/in/jameschristopherosullivan
> > http://www.linkedin.com/in/jameschristopherosullivan
> > Facebook: http://facebook.com/jameschristopherosullivan
> > http://www.facebook.com/jameschristopherosullivan
> >
> > New Binary Press: http://newbinarypress.com/Bookstore.html
> > OpenDAHT: http://opendaht.org/
> > Submit to *The Weary Blues*: http://thewearyblues.org/submit.html
>
>
>
> --[2]------------------------------------------------------------------------
>         Date: Thu, 26 Sep 2013 05:40:59 -0700
>         From: Martin Holmes <mholmes at uvic.ca>
>         Subject: Re: [Humanist] 27.380 peer review?
>         In-Reply-To: <20130926061104.D456D3094 at digitalhumanities.org>
>
>
> I've been doing reviews for the DH conference for many years, and it's
> usually pretty obvious who the authors of most of the papers are. This
> is a small community, and most of us are aware of the specializations,
> projects and preoccupations of most of those who work in the areas we're
> familiar with. If you do any additional reading (following up references
> or checking claims in the abstract) it becomes even more obvious. I
> don't know the motivations of the program committee in making this move,
> but I don't think it changes very much from the point of view of reviewers.
>
> Cheers,
> Martin
>
>
>
>
> --[3]------------------------------------------------------------------------
>         Date: Thu, 26 Sep 2013 15:14:48 +0100
>         From: Melissa Terras <melissaterras at gmail.com>
>         Subject: Re:  27.380 peer review?
>         In-Reply-To: <20130926061104.D456D3094 at digitalhumanities.org>
>
>
> Dear James,
>
> As Program Committee Chair for DH2014, I'm happy to share what I know of
> the
> current situation. It's a very interesting question, and I'm looking
> forward
> to subsequent discussions.
>
> The change to single-blind peer review for DH was made in 2005. In the
> Annexe to the ADHO conference protocol
> (
> http://adho.org/administration/conference-coordinating/annex-adho-conference-protocol
> ),
> you can see, under "Reviewing of Papers", "ADHO is discontinuing the
> practice of anonymizing papers during the review process." It would require
> some digging to get the minutes of the discussion that let to it, but as I
> understand it, this was because author identities are often so recognizable
> from the DH projects referenced in proposals.
>
> The conference organisation is bound by these protocols from one year to
> the
> next - so the procedure for DH2014 is now set up. However, we of course
> welcome input and discussion, and if there is a sense from the community
> that this needs to be changed, this can be addressed at committee level,
> and
> the protocols amended. Our community has changed a lot since 2005, indeed.
>
> The Conference Coordinating Committee would be the place to direct any
> recommendations, or queries, or to ask for this to be raised again as a
> discussion item, as they are responsible for maintaining the conference
> protocols. http://adho.org/administration/conference-coordinating
>
> We welcome any members of the ADHO community asking this sort of question
> about process, and as our community grows and develops, its important to
> think about how existing protocols reflect the community's needs.
>
> best,
>
> Melissa Terras
> (PC Chair DH2014).
>
>
>
> -----------------
> Melissa M. Terras MA MSc DPhil CLTHE CITP FHEA
> Director, UCL Centre for Digital Humanities
> Reader in Electronic Communication
> Department of Information Studies
> Foster Court
> University College London
> Gower Street
> WC1E 6BT
>
> Tel: 020-7679-7206 (direct), 020-7679-7204 (dept), 020-7383-0557 (fax)
> Email: m.terras at ucl.ac.uk
> Web: http://www.ucl.ac.uk/infostudies/melissa-terras/
> Blog: http://melissaterras.blogspot.com/
> Twitter: @melissaterras


-- 
*James O'Sullivan *
@jamescosullivan  http://twitter.com/jamescosullivan **
Web: josullivan.org

Twitter: http://twitter.com/jamescosullivan http://twitter.com/#%21/jamescosullivan 
LinkedIn: http://linkedin.com/in/jameschristopherosullivan http://www.linkedin.com/in/jameschristopherosullivan 
Facebook: http://facebook.com/jameschristopherosullivan http://www.facebook.com/jameschristopherosullivan 

New Binary Press: http://newbinarypress.com/Bookstore.html
OpenDAHT: http://opendaht.org/
Submit to *The Weary Blues*: http://thewearyblues.org/submit.html





More information about the Humanist mailing list