Viewing entries tagged
web services

1 Comment

Designing well-formed URIs for your REST Web API

apiThis is a follow up post to “Using HTTP Status Codes correctly in your REST Web API”, and “Using HTTP Verbs correctly in your REST Web API”. Continuing with the theme of RESTful Web APIs, I thought I would touch on the importance of the actual URL/URI part of the Web API. One way we can think of our Web API is as a database exposed through the HTTP layer. Like any good database, we must be careful when designing "the DB model of our Web API". The equivalent to a formal DB Model (ERD for relational databases), is the Resource Modeling that goes into the Web API. Let’s dive right in.

In REST lingo, URI endpoint are referred to as a Resource. From a data standpoint one can think of it as a piece of data describing an entity instance. Here’s some housekeeping rules to keep your Resources neat and clean:

  1. Avoid having your API start at the root domain. The two most common practices are to create a subdomain (such as http://api.mydomain.com), or to have a dedicated base URL different from the root domain (like http://mydomain.com/api). In doing so you ensure the longevity of the URIs assigned to your resource endpoints, and avoid potential collisions with Page URLs you may want to name in certain ways for the website hosted on your root domain. It also helps later on when versioning becomes an issue for a matured Web API.
  2. The resource endpoints should be plural and NOT singular, for example:
    • http://.../api/customers
    • http://.../api/orders
    • http://.../api/products
  3. Use identifiers to locate single elements in the resource URI. An identifier should ALWAYS resolve to the same single resource.
    • http://.../api/customers/123
    • http://.../api/orders/abc77uk
    • http://.../api/books/world-war-z
  4. When designing your resources, the golden rule is “Nouns are GOOD, Verbs are BAD”. Do not use verbs in ANY part of a resource URI.
    • If you see a verb as in a URI, like http://.../api/getcustomers or http://.../api/payfororder, a part of you should die.
    • Do everything in your power to change it, and educate the creators why using verbs is a bad practice.
  5. For non-resource URIs (yes, they do exist) make sure they are CLEARLY functional endpoints.
    • A simple way to do this is to have a special URL path for all functions such as:
      • http://.../api/func/calculate-fees?size=5&weight=8
    • Please don't use non-resource URLs as an excuse to build a RPC style API.
    • Other samples of functional endpoints could be:
      • http://.../api/func/calculateTax?state=fl&amount=10
      • http://.../api/func/randomNumber
      • http://.../api/func/getHash?input=ubniq2

 

As I'm writing this post I cannot think of anything else to add, but this is a good place to start when it comes to the design of your Resource Model for your REST Web API.

Happy coding!

1 Comment

2 Comments

Using HTTP Verbs correctly in your REST Web API

Following up after the earlier post titled Using HTTP Status Codes correctly in your REST Web API, here is one on using the HTTP Verbs from the W3C spec in the "right way" to have a clean REST Web API. Just like the HTTP status codes, there are many more verbs in the HTTP standard that, although they are in theory OK to use for your Web API, you can get by with just a few that helps to keep your API simple, and self-explanatory to its clients.

The full list of HTTP verbs from the spec can be found HERE, but we are going to focus on how to interpret the verbs and the actions that should be executed for each on in the context of a well-defined REST Web API. In the table there is also the result set that standard clients expect when they make requests with such VERBs. To better understand their proper use, we'll use a sample resource endpoint called Users, where the (you guessed it) "Users" of our app are exposed via our Web API.

Resource Sample GET (aka Read) POST (aka insert) PUT (aka update) DELETE (aka delete) PATCH (aka partial update)
api/users Action Gets a list of users Creates a user Batch Update Errors out Batch Update the users only with the attributes present in the request
Return List of users New user No payload, only HTTP Status Code Error HTTP Status Code No payload, only HTTP Status Code
api/users/123 Action Gets a single user Errors out Updates the user Deletes the user Partially updates the user only with the attributes present in the request
Return Single user Error HTTP Status Code Updated user No payload, only HTTP Status Code Updated full user object

 

And this is how you properly use the HTTP Verbs in a REST Web API.

Happy coding!

2 Comments

Comment

Using HTTP Status Codes correctly in your REST Web API

There are like a gazillion HTTP status codes maintained by the W3C and the Internet Assigned Numbers Authority (IANA) in their Official Registry for the HTTP specification. For RESTful Web APIs, even though in theory you could use any of them if the occasion deserves it, I've found that simplifying their use helps in making your API self documenting in nature and simplifies the cases your Web API clients need to consider. Here is my list of 'useful' HTTP Status Codes and how your clients can/should interpret them:

Code Description What it really means for a client of the Web API
200 OK It worked!
201 Created The resource was created OK!
304 Not Modified The client can use the cached version of this resource, because nothing has changed.
400 Bad Request The client did something wrong. The request has bad syntax or cannot be fulfilled.
401 Not Authorized The Web API is requesting the client to authenticate.
403 Forbidden The server understood the request, but is refusing to fulfill it due to restrictions in the client's authorization. Do not try again.
404 Not Found The resource was not found. There is nothing on that endpoint URI.
500 Internal Server Error The author of the service did something wrong. Something went bad on the server. (IOW: the Web API is fucked up)

 

I always include a similar table for my API guidelines page (note I didn't say documentation, cuz a well designed REST Web API should be self documenting)

Happy API designing!

Comment

Comment

A Message to "Enterprise Architects"

Dear Enterprise Architects: The Web was built on a RESTful architecture. Stop trying to invent fancier systems for distributed computing (wink to the herds of ESBs out there).

There isn't a bigger, more proven, more performant, more scalable specimen of distributed computing than the Interweb.

That's right, less is more. Fuck everything else for distributed computing and messaging, even for the "Enterprise".  You need to un-train yourself, stop being a baby and embrace it already, it is the year 2012.

And relax, your Enterprise applications will scale just fine. Now have some cheez,

Best,

Michel Triana

 

World Wide Web - Enterprise Architects

Comment

Comment

WCF WTF!

It gets me when application frameworks tamper with core web concepts of precisely what they are trying to solve. If you have WCF services exposed through any of its different endpoints, you have to do the most ridiculous dancing to get something as simple as the HttpContext. WTF is up with that Microsoft!?!

There are like 10 different ways to access HttpContext and Request Headers, all weird in their own ways, none of them standard, and requiring the callers to add headers in different and specific ways:

  • There is HttpContext (or this.Context or HttpContext.Current): “Gets or sets the HttpContext object for the current HTTP request” This would be the obvious choice, but the WCF team needed to get COMPLICATED! To support this, you have to add extra magic and attributes to your service contracts (read here)
  •  Then we get fancy with something that is not quite the HttpContext the WEB knows and loves, but some new BS called OperationContext (OperationContext .Current). MSDN explains: “Provides access to the execution context of a service method”... but off-course!
  • Also HttpContextBase class according to MSDN “serves as the base class for classes that contain HTTP-specific information about an individual HTTP request”. So, you’d only think that HttpContextBase is the base class of HttpContext right? WRONG!

Hmmm, at this point you think this might be a brain teaser. There may be another 2-3 ways to access data from a similar concepts. If inspecting the HttpContext on the server side is a nightmare, managing headers and contextual http request elements on the client is even worse if your client is using the generated WCF contracts from VS. Here you are either setting something called ‘OutgoingMessageHeaders’ on an http request (like there is something that can be ‘incoming’ during a request), or you are implementing a custom IClientMessageInspector and altering the request before it is sent to the server: what is this the Police Academy (Inspector, pffff)? Why do I need to inspect a message I built? Or why am I forced to do this kind of crap?

This is so frustrating I cannot cope with the unnecessary layers of engineering and noise the WCF team threw over such a simple concept. I have nothing against new and different ways to solve problems, but please don’t call it the same as something that already exists and it’s well defined by the HTTP protocol specification (RFC 2616). PLEASE. DON'T.

I’ll try working around it with Rick Strahl’s post. If I keep having problems, I’ll move out to a different framework, implement my IHttpHandler, or downplay WCF’s capabilities.

Comment

Comment

OWASP Guidelines, what you should know.

The Open Web Application Security Project (OWASP) is an open-source application security project with a community that includes corporations, educational organizations, and individuals from around the world. This community works to create freely-available articles, methodologies, documentation, tools, and technologies. The knowledge and guidelines on this document are meant to be best security practices all web designers and web developers should follow to ensure their web applications are not vulnerable to common attacks and their users are kept safe. If you are a software developer and you write web applications you should take OWASP guidelines very seriously and make it part of your SDLC and code review process, if not there already. As with any other guideline, this is a living project and threads evolve everyday, so the guideline evolves everyday as well. Please be sure to read and keep yourself up to date with the most recent and accurate information about security standards and guidelines through the information provided in the OWASP website, online forums, papers, and various magazines that often publish security articles such as MSDN, VS Mag and Communication Arts.

OWASP has a project called 'The OWASP top ten project'. This project publishes every 2-3 years the top ten threads, security flaws and attacks web applications are victim off. All developers and project managers should take time to read and understand this list. If you discover new flaws that have not been managed previously in your web applications, do not waste time to discuss it with your peers and come to a resolution sooner rather than later. Entire companies have gone bankrupt or lost a valuable market share because they didn't pay enough attention to the security on their website and information was either compromised or completely gone forever due to hacker attack.

The OWASP List

This is a compressed list of my top 12 issues, they are common threads and attack strategies followed by hackers and how developers should prevent them. They are comprised by the last top 10 from the 2010 list, and another 2 from the 2007 list. Most of the 2007 list are repeated throughout the top 10 of 2010. They all have references as to what they are with a direct link to the OWASP site that contains more details about each thread.

1. Cross-site scripting (XSS) -- Applies to: 2007-A1, 2010-A2

  1. Avoid JavaScript running from 3rd party sites.
  2. If using jQuery and have CDN's sources, make sure the declaration is written on each http page and your CDN is a popular one, like the Microsoft or Google CDNs.
  3. Avoid places that allows embedding JavaScript code. If the web application has input controls, make sure they are ALL fully validated. Parse escape characters and other entries that are not allowed in any of them.
  4. Try to ensure your web application ignores all outside domain JavaScript remote procedure calls.
  5. By using SSL and the aforementioned JavaScript security checks, conditional XSS will never happen.
  6. Web applications using WCF technology should always use a secure binding such as netTcpBinding or ws-HttpBinding with transport layer security.

2. Malicious file execution -- Applies to: 2007-A2

  1. NTFS file permissions prevents unwanted file execution.
  2. Make sure the database process (SQL Server, MySql, Oracle) is NOT running under a local system account, but instead it runs always under an account with limited credentials. Never leave the database process running under full trust account like 'sa'.
  3. For ASP.NET deployments, make also sure the ASP.NET process does not run under a local system account, but instead it runs always under an account with limited credentials explicitly created for your ASP.NET process.
  4. Disable file execution ALWAYS on your database server (particularly important for SQL Server and Windows Server deployments).

3. Insecure direct object references -- Applies to: 2007-A4, 2010-A4

  1. Never show internal identifiers of customer records, tables keys, database object names and transactional records identifiers in any part of the web application, including the address bar. Potentially compromising data should not be exposed on the client side.
  2. Never expose object references to be consumed by the users. This should be an internal rule and should not be violated. In the case that the obvious approach is to expose object references, use tokenization and internal private mapping instead.
  3. Make penetration testing is part of the beta-test for all web applications prior to stable release to the public.

4. Cross-site request forgery (CSRF) -- Applies to: 2007-A5, 2010-A5

  1. If you have authentication on your site and have pages where  any financial or sensitive information is transmitted, you should always require access to such pages through TLS/SSL.
  2. If your application is a ASP.NET applications, let ASP.NET maintain the ViewState. This is a good approach to prevent an attacker to forge a valid Viewstate. ASP.NET controls vulnerable to CSRF attack are run in the server runat="server"
  3. The reasons for an attack of this type are very similar to those for a XSS attack. Please read how we prevent before in this post.

5. Information leakage and improper error Handling -- Applies to: 2007-A6

  1. Your web application should never exposes debugging messages, stack traces or path information to the browser. Instead, you should create a friendly application wide Error Page and 404 Page to be render in the browser in case of unexpected behavior or exceptions occur.
  2. Never expose private information of your users to another user. This is a common error among developers, where they try to prevent user duplicates and end up sharing more than it is needed to a prospect new user of your web site... "Are you John Smith from 555 Pen Ave?" NEVER DO THAT.
  3. Clear all sensitive information on every post back. Things like credit card information, PAN, Address, Billing, etc should not be persisted in between post-backs. Use JavaScript to validate this input fields to prevent full server round trips and ease user frustration when making errors on those pages.
  4. Internal DB identifiers should never shown to the user in the application: "Your customer id is 56457" and that is the actual primary key of the Customers table. NEVER DO THAT.

6. Injection flaws: SQL injection, LDAP and Xpath injection flaws, other injection flaws -- Applies to: 2007-A2, 2010-A1

  1. Filter escape characters of known SQL injection attacks. Use validation schemes to ensure data going into queries does not contain any attacks.
  2. Always remove bad or unsafe user input by means of sanity methods before executing any queries in your databases. There are plenty of good sanitation algorithms and methods on the web for this purpose.
  3. Use parametrized queries and store procedures are used when interacting with databases.
  4. Use a DAL implementation that relies on industry best practices. For example, if building web apps with ASP.NET/C# you can use the fantastic Microsoft Enterprise Library or Entity Framework to ensure consistency, security and best practices.
  5. Enforce least privileges when connecting to databases and back-end systems.

7. Broken authentication and session management -- Applies to: 2007-A7, 2010-A3

  1. If end users must change passwords, they have to re-authenticate, even if they have a valid session id.
  2. If a web application provides public authentication to end users, provide a locking mechanism where users are restricted to a maximum of 5 failed attempts of authentication per hour/day/whatever. Have something for this and log the failed attempts.
  3. Never store clear text passwords in your database of configuration files. You can use simple proven cryptographic solutions like the use one-way password hashing.
  4. User authentication credentials should be protected by transaction using SSL in all login and account management pages.
  5. Also refer to XSS and CSRF entries.

8. Insecure cryptographic storage -- Applies to: 2007-A8, 2010-A7

  1. Always encrypt sensitive data using industry standards cryptographic algorithms approved by National Institute of Standards and Technology (NIST) and the NSA.
  2. In the unavoidable scenario that we need to store sensitive data in resident memory for a short period of time, DO NOT use traditional native types of the language of your choice. Most languages provide marshaling types that provide security of data while in memory. If using C# you can use the SecureString for memory persistence. This will prevent a hacker from doing a full memory dump on the target PC and grabbing all the clear text content you had saved in memory before.

9. Transport Layer Protection -- Applies to: 2007-A9, 2010-A9

  1. Whenever possible, have communication between clients and the servers always goes through Transport Layer security.
  2. Each request to the web server is individually authenticated and authorized.
  3. All pages available through a TLS/SSL are NOT available also over a non-SSL connection. IF YOU HAVE IT ONE WAY, PLEASE AVOID THE OTHER.
  4. Cookies are marked with the "Secure" flag.
  5. Sensitive data, reference objects and private identifiers should never be exposed through URLs.
  6. Get TLS/SSL certificates by a popular Certificate Authority, like VeriSign or Thawte

10. Failure to restrict URL access -- Applies to: 2007-A10, 2010-A8

  1. Consistently enforce access control in presentation layer and business logic for all URLs.
  2. Role based authentication is provided as a minimum for web applications with configurable policies. Authentication credentials should never be hard coded.
  3. Authorization is verified by the web server prior to delivering responses to any web request.
  4. Anonymous users should only have limited access to registration pages or general information pages if any.

11. Security Misconfiguration -- Applies to: 2010-A6

  1. Updates to all servers, OS, Web/App server should be planned ahead of time and notified to the clients and development team to assess possible impacts of the updates.
  2. Default Administrator passwords are immediately disabled after a new OS is installed (if needed).
  3. Maintain constant communication and give special attention to Framework updates, database servers updates, OS updates and any patches, security corrections or changes that affect each other.
  4. Stick with a Framework. If you are using .NET, embrace it. The use of .NET Framework provides a stronghold architecture to build software. The same is true for other fantastic frameworks. Having too many frameworks means you need to focus on more content to maintain, which ultimately becomes very hard unless you have dedicated teams for it.
  5. Prior to any roll make a detail testing of the applicable updates in conjunction with applications. The same process should happen for new application updates and patches. Depending on the complexity of the application or update being deployed, more time may be needed. Do not rush the deployment of applications due to deadlines, we are better safe than sorry.

12. Unvalidated Redirects and Forwards -- Applies to: 2010-A10

  1. Avoid using redirects and forwards.
  2. In the inevitable need of using redirects, be sure you enforce authorization of parameter values used by the end users. Instead of actual values, protect your redirects with server mappings for the parameters passed in.

That's it. Obviously this was a rapid overview to the most common threads. You can find a lot more at www.owasp.org with a lot of good advice as well.

Happy coding!

Comment