I have just submitted two reviews for the ACL 2025 conference, and overall, I would say it was a positive experience. It was partially an eye-opener and also a confirmation of some problems in our current reviewing system. So far, my whole experience with reviews has been from two dimensions:

1. As an author, I received a few reviews—some were informative and insightful, while others were unfair and low-effort.
2. As an “evaluator,” as this is my current research topic—Peer Review Evaluation.

What’s new this time is that I got to write reviews. In the beginning, I was anxious, as this was something new to me, and I didn’t know how to approach it. But the ACL Reviewer Guidelines do a great job explaining what should and shouldn’t be done. Another burden was that this is my area of research—“How to Write Good Reviews?"—so I felt the responsibility to apply everything I have been promoting in my reviews.

What is Peer-review

Peer-review is a corner-stone in scientific research. It serves a quality check for scientific drafts. It is formally defined as: Peer review is the evaluation of work by one or more people with similar competencies as the producers of the work.

Why is it important

It’s important as, only after an article passes peer review can it be officially published, and it’s the knowledge shared with the scientific community and the world.

You and your Research

https://www.cs.virginia.edu/~robins/YouAndYourResearch.html

Main Ideas

• Commit to your idea
• Ask yourself: What are the important problems in my field?
• Communicate with the bright minds
• Ask the important questions
• Closed door, or open door
  • when you work with door closed
    • you work harder
    • you work on the wrong thing
  • open door working
    • get many interruptions
    • get important clues
  • My take on this is we should keep an open mind about what other people are doing and where the research is heading

Intro

Here, we are going to discuss the internals of PyTorch Autograd module. The most of us don’t have to know about this. I was the same till I came across this error:

unsupported operand type(s) for *: 'float' and 'NoneType'

This came from executing the following code:

import torch
a = torch.tensor(5.0, requires_grad=True) * 0.1
b = torch.tensor(2.0, requires_grad=True)
c = a + b
c.backward()
a += + 0.1 * a.grad

But why? We defined that the gradient of a should be calculated by putting requires_grad to be True!

TTS (Text To Speech)

TTS can be viewed as a sequence-to-sequence mapping problem; from a sequence of discrete symbols (text) to a real-valued time series (speech signals). A typical TTS pipeline has two parts; 1) text analysis and 2) speech synthesis. The text analysis part typically includes a number of natural language processing (NLP) steps, such as sentence segmentation, word segmentation, text normalization, part-of-speech (POS) tagging, and grapheme-to-phoneme (G2P) conversion. It takes a word sequence as input and outputs a phoneme sequence with a variety of linguistic contexts. The speech synthesis part takes the context-dependent phoneme sequence as its input and outputs a synthesized speech waveform.

Distributed Training

Why?

• Need more compute power to process large batches in parallel (DDP)

  • Uses collective communication

• Large model that couldn’t be fit in memory of one GPU (RPC)

  • Uses P2P communication

• All of the above XD

DDP in Pytorch

• Every GPU has a model replica, controlled by a process.
• Every process fetches different batch of data.
• Forward.
• Overlapping between computation of and communication(broadcast - allreduced) of gradient.
• Validation