Preface
In my previous blog, Part 1, I went over what I did "wrong" during the development of Ping Collector. In Part 2, here's what I did *sorta* right.What I did Right
-
I finished the project.
- Google App Engine
- Memcache
- Non-relational database use (Google Datastore)
- Database design around a non-relational database.
- Cross-environment Symmetrical Encryption (Implementation of algorithms, not the algorithm itself)
- Hashing
- Request Limiting
- Client authentication (Is the data I am receiving on my server by a permitted client?)
- User Authentication (What user is the data I am receiving coming from? Custom implementation.)
- Windows Services
- Various platform support for the Client and its Installer (.NET 2.0+, Win XP - Win 8)
- InstallShield Express with various custom actions to accomedate Service installation and Windows 8.
- Encrypted Storage of credentials on Windows environments.
- Retrieving/Storing values in registry (Windows)
- Log4Net
The "Request Gateway" method which was once implemented as Before Advice for all methods before it was decided that Spring Aspect Oriented Programming was too complex for the Google App Engine environment. public void requestGateWay(HttpServletRequest req,
HttpServletResponse resp, int requestCooldown, String functionName)
throws TooManyRequestsException, UnsupportedISPException {
System.out.println("REQUEST GATEWAY");
String isp = ispService.getOrg(req.getRemoteAddr());
if (isp != null && testingBypass == false) {
if (!isp.toLowerCase().contains("windstream") && testingBypass == false) {
throw new UnsupportedISPException("Only Windstream internet users are supported currently.");
}
}
// may god have mercy on your latency.
Long ipExpiration = (Long) cache.get(req.getRemoteAddr() + functionName);
if (ipExpiration == null) {
System.out.println(req.getRemoteAddr() + "'s requests limited for "
+ requestCooldown + " minutes after method call");
Expiration expir = Expiration.byDeltaSeconds(requestCooldown * 60);
cache.put(req.getRemoteAddr() + functionName,
expir.getMillisecondsValue(), expir);
} else {
Date expirDate = new Date(ipExpiration);
PrettyTime pt = new PrettyTime();
System.out.println("IP Address " + req.getRemoteAddr()
+ " requested too frequently. Next request available in "
+ pt.format(expirDate));
resp.setStatus(429);
throw new TooManyRequestsException(new RequestLimitInfo(false,
"You are requesting the feature, "" + functionName
+ "" too frequently. Please try again in "
+ pt.format(expirDate) + "."));
}
}
-
Google's (NoSQL-like, non-relational, BigTable-based) Datastore was a great choice.
A humorous, early test snippet generating a piechart based on the data collected. public void testPieChart(HttpServletResponse resp,
@RequestParam(required = true) String user, @RequestParam(required = true) String sauce,
@RequestParam(required = false) boolean peakTime) throws IOException,
NoPingsFoundException {
if (!secretSauce.equals(sauce)) {
resp.sendError(403, "You didn't provide the secret sauce.");
} else {
Filter filter = null;
PieChart chart = PingPieGraph.generatePingDistributionChart(PingDAO
.getAllPingsBetweenDates(user, filter, peakTime));
String title = "Report for " + user;
if(peakTime == true){
title += " during Peak Time (4pm-12am EST)";
}
title += " Date: " + new DateTime().toString("dd/MM/YYYY"); ;
chart.setTitle(title,Color.BLACK,10);
System.out.println(chart.toURLString());
resp.sendRedirect(chart.toURLString());
}
}
-
Taking it relatively slow.
Obfuscating model classes which are serialized WILL make you have a bad day [System.Reflection.ObfuscationAttribute(Feature = "properties renaming")]
class PingCollectorResponse
{
[JsonProperty(PropertyName = "message")]
public String message { get; set; }
[JsonProperty(PropertyName = "code")]
public int code { get; set; }
[JsonProperty(PropertyName = "error")]
public Boolean error { get; set; }
}
Conclusion
Some other non-technical thoughts:- Start small, work up. The first iteration and alpha version of Ping Collector simply sent emails with the latency data, and it was made during my short vacation after falling six feet onto my back in a unfortunate crowd surfing accident (I love my metal, a little too much). Gradually, features were added and goals were set, and Ping Collector is where it is today.
- If you lack the skill or confidence to build something, put monetary goals aside.
- StackOverflow is a godsend.
- Program things for your mother. She helped pay for your education.